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PREFACE 
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This report describes a study performed for NASA Ames Research 
Center on methods for predicting the cost, reliability, and maintaina- 
bility of advanced avionics systems designed for use by general avia- 
tion in the 1980s and beyond. The study is one of a number of NASA- 
sponsored research efforts encompassed by the Advanced General Avia- 
tion Avionics System (AGAAS) program. The purpose of the Rand study 
is to provide NASA with information about avionics cost, reliability, 
and maintainability (CRM) that will be helpful in formulating the 
succeeding phases of the AGAAS program and in evaluating alternative 
technical approaches proposed by participating contractors. Practical 
problems of predicting the CRM of advanced avionics systems for gen- 
eral aviation are examined in detail. The usefulness and shortcomings 
of the different modeling approaches for cost and reliability estima- 
tion are discussed, together with the special problems caused by lack 
of historical data on the cost of maintaining general aviation avion- 
ics. Suggestions are offered on how NASA might proceed in assessing 
CRM implications of advanced avionics in the absence of reliable gen- 
eralized predictive models. 

This study draws heavily on results of earlier, government- 
sponsored research in the area of cost and reliability predictions and 
maintenance requirements. Air Force and airline experience is 
presented wherever it is deemed relevant to advanced general aviation 
avionics . 

Views or conclusions expressed herein are the authors' and do not 
necessarily represent the opinion of NASA. This Working Note is 




intended to transmit research results to the sponsor, and may not be 
distributed without the sponsor's approval. 
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SUMMARY 


General aviation forms a large and important segment of the total 
U.S. civil aviation activity. It accounts for the vast majority of 
aircraft operations, includes more than 95 percent of the total U.S. 
civil aviation fleet, and contributes a substantial fraction of the 
passenger miles flown each year. General aviation has played an 
important role in the economic growth in the U.S., and it has the 
potential to play an even greater role in the future. 

There is, however, significant uncertainty about the future of 
general aviation in the U.S. because of the unknown impact of the 
increasingly complex and costly equipment that the Federal Aviation 
Agency (FAA) requires in order to operate in the National Airspace 
System. Some effects are evident already: operating procedures are 
complicated, regulations are becoming more and more restrictive and 
comprehensive, and the requirements for new and/or better avionics 
equipment are drastically increasing the cost of owning an airplane. 
A related effect is the increasing demand on pilot training and profi- 
ciency necessary to operate the additional equipment efficiently and 
safely. 

To address these problems, NASA recently initiated a program of 
Advanced General Aviation Avionics Systems (AGAAS). The objective is 
to provide the critical information required for the design of a reli- 
able, low-cost, advanced avionics system (AAS) which would enhance the 
safety and utility of this mode of transportation. 

The objective of the present study is to provide NASA with a 
methodology for estimating cost, reliability, and maintenance require- 
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ments (CRM) for advanced general aviation avionics equipment operating 
in the 1980s. In the course of the study, all known research done by 
others in CRM prediction methodologies for avionics and related equip- 
ment was reviewed, and visits were made to four major manufacturers of 
general aviation avionics equipment and to a number of Air Force agen- 
cies to learn how they handle the problem of CRM prediction. 

Our analysis of past attempts at creating generalized parametric 
models for predicting cost and reliability revealed that these efforts 
have produced limited and not very useful results. The utility of the 
models is inherently limited to prediction within a specific technol- 
ogy, and they are unsuccessful in prediction where the technologies 
have changed from those used to formulate data for the original 
models. All avionics equipment is in a period of rapid technological 
evolution, so that parametric models based on experience with current 
equipment are seldom useful in predicting cost and reliability charac- 
teristics of future equipment. Furthermore, electronic equipment is 
peculiar in that a desired function can be produced by a wide variety 
of electronic means, each of which has its own circuitry and associ- 
ated parts. Devices with similar, if not identical, performance capa- 
bility can have very different internal components, and therefore very 
different manufacturing costs, maintenance problems, and reliability. 

Avionics for general aviation poses a special problem in that 
little or no systematic data are collected on the actual reliability 
or support cost of the equipment. Therefore, it was not possible to 
develop any parametric models, however limited, in response to the 
basic objective of the present study. However, the research produced 
a substantial amount of information relevant to the design of future 
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avionics for general aviation, and that information is summarized in 
this Working Note. 

Our survey of manufacturers of avionics and related equipment 
revealed that they do perform cost, reliability and maintainability 
(CRM) analysis, but generalized parametric models are not considered 
to be trustworthy and are therefore not used. Although most manufac- 
turers hold the details of their CRM prediction methodologies as 
proprietary, the general approach is engineering based, and not 
parametric. This engineering approach requires a preliminary design 
sufficiently detailed to allow identification of specific parts and 
fabrication methods for use in cost prediction and 
reliability/maintainability assessment. Most reliability assessment 
programs are a company version of MIL Handbook 217B, modified to 
include company experience and methods. The use of such methods will 
allow the cost data for one system to be compared with the cost data 
of other systems. The ability to perform sensitivity analysis, how- 
ever, will be limited or nonexistent. 

In the absence of generalized, parametric assessment techniques, 
NASA should require that each submission of a candidate design be 
accompanied by a contractor's prepared estimate of users' purchase 
price, maintenance philosophy, and maintenance cost estimates. These 
estimates should be backed up by a description of the cost-estimating 
methodology used, and a listing of analogous and corroborative data. 
The description should be complete enough that the estimates can be 
duplicated and evaluated. Estimates prepared in this way will most 
likely be engineering estimates that rely on piece parts count, piece 
parts prices, and manufacturing labor cost estimates. 



The inputs to these cost estimates can be reviewed for their rea- 
sonableness and completeness, or in the case of technologies that are 
not yet commercialized, for evidence that the estimate made is defen- 
sible. Estimates made in this manner can be used to make comparisons 
among the various designs that are submitted, provided that it is 
understood that the estimating methods used for different levels of 
design will probably have widely differing (and unknown) estimating 
accuracy. Therefore, comparisons that are made should stress general 
cost ranges rather than exact differences between estimates, and 
should concentrate where possible on uncertainties in the estimates. 

The closely related characteristics of reliability and maintaina- 
bility pose a particularly troublesome issue in the evaluation of AAS 
designs. General aviation equipment is now supported and maintained 
by a wide variety of organizations, many of them relatively small. 
Yet the AAS designs seem likely to involve a highly integrated system 
of digital components, posing a very sophisticated maintenance prob- 
lem. The Air Force and airline experience with advanced avionics 
equipment shows fault diagnosis and isolation to be the number one 
maintenance problem. The design of tests (i.e., diagnostic program 
development) for detection and location of faults in highly integrated 
systems is often frustrated by great difficulties in predicting 
failure modes. For this reason it is essential to emphasize reliable 
means of fault isolation early in the design stages. To ensure that 
maintainability is considered early in the design, a requirement must 
be placed on the designer to prepare a maintenance philosophy and plan 
as part of the preliminary design. An evaluation of the philosophy 
and plan should be conducted at each design review to ascertain if 
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they will work, given the technical capabilities of existing mainte- 
nance personnel and the organizational structure through which mainte- 
nance is most likely to be performed. 

It appears that introduction of AAS-like equipment will require 
some major changes in the organization of the support and maintenance 
system for general aviation avionics, as well as in the design of the 
equipment. Thus it is unlikely that parametric models based on past 
experience will be suitable for reliability prediction of the new 
equipment. Moreover, the estimation of absolute reliability in prac- 
tical terms for new equipments, using new parts and untested designs, 
is extremely difficult. Nonetheless, the prospects for a more limited 
assessment of reliability--the comparison of two (or more) candidate 
designs for their inherent relative reliability — are quite favorable, 
given only a few easily understood standard procedures. First, use of 
the standard MIL Handbook 217B methodology gives assurance that many 
of the factors known to affect reliability are being assessed equally 
between or among candidates. The use of a newly available digital 
computer program for this purpose increases the assurance of complete- 
ness and lack of bias (both essential to producing the best of what 
may still be uncertain predictions), and at the same time reduces the 
effort and expense required to provide this vital comparative measure. 

The specific procedures consist of: 

1. Listing the individual electronic parts of the system, 
which means that a detail design must exist. 

Associating all relevant parameters with each part, 
such as operating conditions, quality level, etc. 


2 . 
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3. Identifying special characteristics of parts (number of 
gates in an IC, voltage rating of capacitors, etc.). 

4. Applying the above information to tables in MIL-HDBK- 
217B . 

This detailed procedure does not mean that these modern evalua- 
tion methods cannot be misused to provide extravagant reliability 
estimates. Misuse may come about through the relatively simple 
expedient of minimizing initial design complexity to the point where 
the later addition of circuits and components will become necessary in 
order to meet performance requirements. Even higher reliability esti- 
mates may be had through optimistic parts selection and derating poli- 
cies. It should be clear that such unrealistic assumptions are less 
likely to be excepted now, given today f s standardized estimating 
tools . 

A part of the early design should include a simulation model 
which can be used to test the system performance within its expected 
operating environment; however, since real world performance cannot be 
predicted precisely, it is important to continue the testing and 
updating of this model throughout the design, development, prototype 
testing, and early operational phases of the program. 

Since any AAS will utilize substantial computer programs to per- 
form many of its functions, it is important that potential software 
problems be identified and dealt with early in the design. The prob- 
lems are frequently traceable to the early conceptual stage and often 
result from poor and unrealistic system requirements specification. 
It is therefore important that system requirements be subjected to a 
vigorous review process. 
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Once a good set of requirements is in hand, then the design and 
coding must proceed with great care. Design techniques, organization 
structure, and documentation guidelines must be selected that will 
foster the production of reliable software and the inevitable software 
maintenance that will be required throughout the system's lifetime. 

Although the contemporary approach is toward automatic fault 
diagnosis, there is still a requirement for some degree of human 
interaction in the operation of tests, and in interpretation of test 
results. This human interaction in the fault diagnosis process must 
also be tested during the development process. 
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I. INTRODUCTION 

STUDY BACKGROUND AND OBJECTIVES 

The need for efficient and safe operation of aircraft in high- 
density traffic areas has placed ever increasing demands on the Air 
Traffic Control system. Changes in the Air Traffic Control system to 
meet these demands have necessitated an increase in the cost and com- 
plexity of on-board avionics equipment. This growing complexity has, 
in turn, placed increasing requirements on the proficiency and skill 
required to operate under instrument flight rules (IFR) in heavy 
traffic environments, especially in single-pilot operations. 

In 1975, NASA initiated a program in Advanced General Aviation 
Avionics Systems (AGAAS) with the objective of developing the informa- 
tion needed for industry to produce a reliable, low-cost, advanced 
avionics system that will more effectively integrate the high- 
performance single-engine and light twin-engine aircraft with the air- 
space control system. Emphasis is placed on the use of new technology 
to enhance the utility and safety of aircraft during single-pilot IFR 
operation in high-density traffic environments. Candidate technolo- 
gies include large-scale integrated circuits, microprocessors, new 
sensors, displays, data transfer systems, and advanced system archi- 
tecture. The aim is to relieve the pilot of many onerous tasks that 
can better be done by automated equipment. This will allow the pilot 
to concentrate on flying the airplane and to deal with those aspects 
of flight safety which are better done by a human than by a machine. 

This Working Note describes the study performed by The Rand 
Corporation for NASA Ames Research Center on cost/ 



reliability/maintainability (CRM) for Advanced General Aviation 
Avionics Systems for operation in the 1980s. The study is one of a 
number of NASA-sponsored research efforts [1,2,13,14,20,28], involving 
different contractors, to develop improvements in avionics for general 
aviation aircraft. 

The initial objective of the Rand study was to develop a general- 
ized methodology for estimating CRM requirements for general aviation 
avionics systems operating in the 1980s. This methodology was 
intended to be applied to candidate advanced avionics equipment pro- 
posed to NASA by various contractors, as well as alternative designs 
that might be proposed by NASA. Of particular interest to the study 
were the two preliminary candidate advanced avionics systems (PCAAS) 
designs proposed by Systems Technology, Incorporated (STI), and South- 
ern Illinois University (SIU) . These two systems were to be used to 
initially test the developed methodologies. 

The NASA desire for a method of forecasting CRM for a wide 
variety of systems is understandable considering that they have no 
control over the types of systems that may be proposed. However, the 
broadness of this need places demands upon the methodology that cannot 
be satisfied with the available data and techniques. Previous methods 
were found unsuitable for a number of reasons, and the lack of data on 
current general aviation avionics support costs and reliability 
prevented the development of new methodologies. Consequently, the 
objective of the Rand study was changed to that of (1) analytically 
reviewing the experience of industry and avionics user groups in deal- 
ing with CRM predictions, and (2) drawing from this experience gui- 
dance for NASA on how CRM requirements for candidate advanced avionics 
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systems might be addressed in the absence of a generalized methodol- 
ogy- 

STUDY APPROACH 

A block diagram of a generalized system architecture which was 
used as a guideline for this study is shown in Fig. 1-1. This archi<- 
tecture is sufficiently general in that it adequately matches most 
system designs, including the PCAAS design submitted by STI and SIU. 
The block diagram was useful in identifying most system components 
likely to be encountered in the advanced systems, and it provided a 
framework for thinking about component interaction. / 

j 

In areas where generalized models have been successful in 

/ 

predicting future states and outcomes, the success has been dependent 

I 

on a number of key elements. The first is the ability to identify the 
parameters that affect what is being predicted. The second is /to dis- 
cover or uncover some basic relationship between the parameters that 
describe or characterize the features being predicted. This is nor- 
mally done via physical laws or statistical relationships. '■■■ 

{■ 

In this study our initial task was to identify the general param- 

h 

eters that affect CRM. A number of approaches were used t'o do this. 

f 

First, a survey and analysis of past work in developing generalized 
models to predict either CRM or similar equipment for avionics was 
undertaken. Next, interviews were conducted with manufacturers of 

i ! 

avionics and similar equipment to see just how they dealt with antici- 
pated future technology trends and equipment characteristics (includ- 

i , • 

i 

ing costs), and to find out what they felt were the parameters that 
affect CRM. At the same time, a data collection effort was initiated 

il 

i ,* 

*'? 
i ’• 



Data Buss 



Fig. 1-1 — Typical system organization 
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to acquire appropriate cost and reliability data for the purpose of 
developing generalized cost and reliability prediction models for 
these advanced avionics systems. 

The search for information and data included both current stan- 
dard avionics system components and those predicated on advanced tech- 
nology. It was hypothesized that some system components might remain 
essentially the same as those presently available. These might typi- 
cally be navigation radios, communications radios, transponders, and 
some engine and air data sensors. Certainly hybrid and large-scale 
integrated (LSI) circuit technologies will affect the design of radio 
systems in the mid-1980s; however, there was no attempt made to evalu- 
ate this technological trend quantitatively. 

Components considered to be driven primarily by new technologies 
are displays, microprocessors (and digital interfaces), memories, mul- 
tiple data busses, pilot input devices, and software. Although 
microprocessors and memories are used in some general aviation sys- 
tems, e.g., area navigation (RNAV) , this usage is not widespread and 
not a great deal of data is available. 

Information and data were sought from many sources. Principal 
sources of information investigated were (1) manufacturers of general 
aviation avionics equipment, (2) professional general aviation organi- 
zations, (3) users of general aviation equipment (including Fixed Base 
Operators), (4) manufacturers of general aviation aircraft, (5) Air 
Force agencies, (6) government contractors, (7) principal general avi- 
ation periodical publications, and (8) library literature searches. 

In addition to the above, information on microprocessors, 
displays, and bus structure technologies was sought from manufacturers 
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and users. A complete list of information sources is included in 
Appendix A. 

Using a portion of the information collected, we attempted to 
develop generalized estimating relationships for predicting purchase 
and operating costs of general aviation avionics. For reasons that 
are fully discussed in Section II, these attempts were unsuccessful. 

REPORT OUTLINE 

Although cost, reliability, and maintainability are all closely 
interrelated subjects, they are covered in separate sections in this 
report. Past work in developing cost-estimating relationships, and an 
analysis of cost data collected for contemporary and new technologies, 
are covered in Section II. Conclusions on the development and use of 
generalized cost estimating relationships (CERs) for advanced avionics 
systems for general aviation are included at the end of the section. 
Also included in Section II are recommendations as to how one should 
proceed when generalized methodologies are not available. 

Section III contains a survey of the development of reliability 
prediction methods, a discussion of the accuracies and pitfalls in 
using the different methods, and ways to apply both prediction and 
assessment methodologies where applicable to avionics systems for gen- 
eral aviation aircraft that are projected for operation in the 1980s. 
Section III ends with conclusions and recommendations on how to 
proceed with reliability prediction and assessment in the absence of 
parametric methodologies. 

The subject of maintainability is covered in Section IV. Mili- 
tary and airline experiences are presented, and the applicability of 
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this experience to advanced avionics systems for general aviation is 
discussed. Conclusions and recommendations on maintenance problems 
and philosophy are included at the end of Section IV. 

Some potential software problems that might be encountered in 
developing an Advanced Avionics System are described in Section V. 
The causes of many of the problems are examined, and some generally 
accepted guidelines for avoiding them are discussed. 

Appendix A contains a list of contacts made during the course of 


the study. 
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II. COST ANALYSIS OF ADVANCED AVIONICS SYSTEMS 


INTRODUCTION 

The objectives of the cost analysis portion of this study were 
several. Primarily, methods were desired that could be used to esti- 
mate the costs* of the advanced systems. Originally, it was intended 
to develop estimating methods for both acquisition costs and mainte- 
nance costs. As the study progressed, it became evident that acquisi- 
tion costs could not be easily developed. Since maintenance costs are 
usually a function of acquisition costs, the explicit development of 
estimating relationships for them was dropped. The following discus- 
sion therefore only addresses acquisition costs. 

The objective was to develop cost estimates that were compara- 
tive, in the sense that they could be used to compare the costs of one 
system with another system. They were also to be absolute and as 
accurate as the design of advanced systems would permit, so that the 
costs of any advanced system could be compared with the costs of 
existing avionics systems. As a further objective, the desired 
methods were to be parametric, so that conceptual avionics systems 
could be analyzed from broad descriptive inputs. Thus, the methods 
would express cost as a function of such things as output power, 
equipment weight, number of transceiver channels, etc. With reference 
to the system organization shown in Fig. 1-1, it was hoped that the 
generalized parametric relationships would represent the entire avion- 


*The word cost, as used in this discussion, refers to the cost of 
purchasing an avionics suite, or portions thereof. This "cost 11 to the 
purchaser is equivalent to the "price" of the equipment. 
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ics suite; if this proved difficult, they would be developed to 
represent discrete portions of the suite. Last, the methodology was 
to be able to handle widely differing systems, ranging from those that 
are off-the-shelf to those using the most advanced sensors and elec- 
tronic techniques. 

Past avionics cost studies, as will be seen later, have generally 
not been blessed with the same measures of success that accrued to 
cost studies of nonelectronic hardware. Because of this, and the very 
demanding nature of the study objectives, the cost analysis of 
advanced avionics systems posed special problems. It was hoped that 
some of the problems that had been incurred in previous avionics stu- 
dies resulted from the fact that these studies had dealt almost 
exclusively with military avionics, the special nature of which (e.g., 
military performance specifications and rapid technological evolution) 
narrowed the data base useful for generalized analysis. If this were 
the case, there was the possibility that the characteristics of gen- 
eral aviation avionics would be such that they would be more amenable 
to cost analysis. 

The cost analysis task for the advanced avionics systems (AAS) 
posed some interesting problems. As stated above, the principal 
objective was to develop a generalized methodology by which the costs 
of candidate systems could be estimated and compared. However, a con- 
straint was imposed upon the methodology to the effect that the archi- 
tecture of the AAS and the technology to be used were undefined. 
Therefore, the methodology had to accommodate systems ranging from the 
adaptation of existing off-the-shelf equipment (such as proposed by 
one subcontractor), to systems that were completely redesigned to use 
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integrated electronics and advanced sensors (such as proposed by 
another subcontractor), to systems that were open-ended regarding the 
technology used. 

Costs can be estimated at several stages in the life of a system, 
including a stage prior to its actual design, but different kinds of 
data are required to develop the different kinds of relationships that 
are used. The development of cost estimating relationships (CERs) 
that would apply to avionics equipment of different technologies and 
different states of the art would require that the data base for the 
CERs be composed of these various technologies. Alternatively, 
separate CERs could be developed for each separate statistical popula- 
tion (i.e., each distinct technology and/or state of the art)’ The 
latter would, in this case, require a set of CERs for existing equip- 
ment, and sets of CERs based on future equipment that used new or 
presently unused technologies. This is possible, at least conceptu- 
ally, as long as the technologies or other major characteristics are 
well defined. To the extent that they are open-ended, the task cannot 
be done. Further, the development of CERs for future technologies 
may, of necessity, rely either upon smaller data bases or upon other 
techniques (e.g., engineering analogy) that cannot have the same 
degree of certainty as those used with sound data. This is simply a 
fact of life in the development of CERs, which tend to be good when a 
lot is known about the subject, and less satisfactory when less is 
known. 

Since cost analysts occasionally face this problem, i.e., the 
question of comparing something about which a great deal is known with 
some future replacement about which fairly little is known, how do 
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they deal with it? In part, a situation such as this is not a com- 
fortable one. The ideal analysis is one that relies upon CERs that 
describe a continuum of equipment that includes all types to be com- 
pared. With such CERs, the analyst has a degree of confidence that 
the basic relationships between costs and the parameters contained in 
the CER are well quantified. 

If CERs of this type are unavailable, other means of coping with 
the problem are needed. In the case of the AAS, some of the technol- 
ogy is not new, but is simply used in a new application, i.e., the 
application of LSI circuitry to an entire integrated avionics system. 
For this case, estimates of cost may be made by using an approach that 
breaks the system down into components, estimates their costs, and 
sums them. The success of this method depends on the detail that is 
known about the system and on one's ability to understand the general 
relationships that exist between component part costs and the various 
other manufacturing and marketing steps that lead to a retail price. 
Where the technology is new, and where there are no prior applica- 
tions, costs or prices cannot be estimated by any systematic method. 

In this study, we attempted to develop CERs for the acquisition 
costs of existing equipment, so that AAS designs incorporating present 
equipment could be accommodated. We also examined the new technolo- 
gies expected to be used in other AAS systems and attempted to develop 
CERs for them. Where there were unknowns, due to unknown technolo- 
gies, we have tried to present some perspective on likely costs. 
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SURVEY OF PAST STUDIES ON COSTING 

Any discussion of past cost studies on avionics equipment must be 
prefaced by the comment that virtually all such studies have con- 
sidered military avionics only. Because the cost of military avionics 
is usually substantially higher than the cost of nonmilitary avionics, 
such studies would be expected to be useful in a qualitative sense 
only. From these studies, it would be hoped to discern which equip- 
ment characteristics or descriptors relate to the cost of the equip- 
ment, and the general form of the relationship. Given that the rela- 
tionships are valid ones, it might then be expected that similar rela- 
tionships, with different cost coefficients, would exist for non- 
military avionics, and that the main task facing the cost analyst 
would be one of deriving the correct cost coefficients. 

This hope of being able to "piggyback" on studies of military 
avionics costs is largely a vain one. Of the studies that were 
reviewed, few were found that could be directly related to general 
aviation avionics. In fact, remarkably little successful work has 
been done in the field of military avionics cost analysis, despite the 
fact that all of the cost data ought to be publicly available, and 
that there is an avid interest in the subject. Reasons for this will 
be discussed below, but the general implications of the lack of back- 
ground need to be understood in the context of this study. 

The military studies that have been made largely involve data for 
existing avionic equipment, and were aimed at the creation of CERs for 
that equipment. Thus, the added dimension of attempting to anticipate 
new technologies did not exist to the same degree as it did for this 
study. Despite this possible easier task, few of the studies reviewed 
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were able to generate satisfactory CERs. The implications of this go 
beyond the inability to use military avionics studies as a jumping-off 
place — they say a great deal about the task itself and the prospects 
for successful results. This is especially true because relevant data 
for general aviation avionics is less extensive and detailed than for 
military electronics, and because the technologies that are to be 
expected for the AAS are liable to be new and different from those 
used by the military. 

The major interest in developing military avionics CERs has come 
from the Department of Defense (DoD) ; but the individual services have 
also expressed an interest, usually through contractors. A descrip- 
tion of the studies that have been made follows. To give a more com- 
plete idea of the climate surrounding these studies, not only are the 
studies themselves described, but where no effort has been made, this 
is also discussed. 

It will be seen that success has been rare in these endeavors. 
One of the basic reasons is that electronic equipment is peculiar in 
that a desired function can be produced by a wide variety of elec- 
tronic means, each of which has its own circuitry and associated 
parts. Devices with similar, if not exact, performance capability, 
can have very different internal components, and therefore very dif- 
ferent manufacturing costs. They also can have different maintenance 
problems and costs. Because of the latitude that appears to exist in 
the design and construction of electronic equipment, one of the basic 
premises of developing useful CERs becomes tenuous. CERs are based on 
an assumed relationship between cost and physical and performance 
parameters. If this relationship does not exist, or cannot be 
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uncovered because the data include a wide variety of different basic 
designs, then attempts to develop CERs will fail. The failure will be 
due to the fact that costs are really related not to physical and per- 
formance parameters, which may be well documented, but to circuit, 
component, and construction parameters about which there may be little 
or nothing published. 

Studies by the Department of Defense 

The interests of DoD have mainly been channeled through the 
Office of the Assistant Secretary of Defense, Planning and Evaluation 
(OASD/PA&E), and its predecessor organizations. One study, conducted 
by Resource Management Corporation (RMC), began in 1969, and a final 
report was issued in 1972. The study produced partially useful CERs 
for radar and fire control systems, but CERs for other equipment were 
less successful. For many pieces of equipment, the only significant 
independent variable was weight, which is not a very useful parameter 
for analysis, particularly of conceptual equipment. Basic problems in 
the study concerned data. At the study outset, it was decided that 
collecting the data from contractors involved an effort beyond the 
scope of the intended study. In lieu of this detailed information, 
budgetary and funding data were used which unfortunately contained an 
undifferentiated mixture of R&D and production funds. The poor 
results of parts of the study are partially due to the use of non- 
homogeneous data in developing CERs. 

Rand became specifically involved in the collection of military 
avionics performance, physical characteristics, and cost data for 
OASD/PA&E in 1975, concentrating on the avionics suites of tactical 
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combat aircraft developed since 1965. Data were collected on indivi- 
dual pieces of equipment from manufacturers, the DoD data banks, and 
from the Electromagnetic Compatibility Analysis Center. It was hoped 
that these data could be aggregated in such a way that they could be 
cross-checked with budgetary data for the aircraft systems involved. 
A great deal of data were collected, and some analyses were made. It 
was found that when an attempt was made to create homogeneous samples 
of equipment from which statistical analyses could be made, the sample 
size was reduced to the point where the analysis lost meaning. One of 
the study conclusions was that the addition of data from other types 
of military aircraft might expand the data base sufficiently for 
analysis to produce CERs. 

Rand also performed an analysis of USAF aircraft avionics recov- 
erable spares, and developed a CER. The relationship is based on the 
number of aircraft and the avionics subsystem flyaway cost, and has an 
input that is a technology indicator. 

General Research Corporation developed a complex parametric CER 
for ground based radars. The CER uses about 30 inputs, many of which 
require detailed knowledge of the device. This is not an avionics 
CER, but it is interesting from two points of view. It is electronic 
equipment, as are avionics, and its success is partly due to the large 
number of detailed inputs that are required. In this way, it 
approaches the concept of an engineering cost estimate, where detailed 
knowledge of the components is required. 

Planning Research Corporation conducts military analyses that 
sometimes require avionics cost estimates. They generally make such 
estimates based on analogous systems. The extent to which estimates 
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of this type are successful depends on the type of analysis that is 
involved. Where the avionics costs themselves are not under analysis, 
they are probably satisfactory. However, if the object is to examine 
(for example) the cost effects of varying the parameters of the avion- 
ics themselves, these types of estimates are unsatisfactory. 

Air Force and NASA Studies on Avionics CERs 

The Air Force Systems Command has been interested in avionics 
CERs in its Aeronautical System Division (ASD) and its Electronics 
System Division (ESD) . Reliable, formal avionics CERs have not been 
developed in either division, however. Further, since most of their 
needs are for relatively near term equipment, they have been able to 
use the RCA PRICE model described more fully on p. 18 [18]. Typical 
applications of avionics costs include the life-cycle cost analysis of 
projects such as DAIS,* in which time phased acquisition and O&M costs 
are compared for competing pieces of hardware to determine minimum 
life cycle costs. DAIS uses a well developed life cycle cost metho- 
dology, but the equipment costs inputs to this methodology are sup- 
plied by the equipment contractors. These, in turn, develop their 
cost estimates by industrial engineering techniques using prototypes, 
or they use the PRICE model. 

General Research Corporation (GRC) has also done work for the Air 
Force Avionics Laboratory on developing CERs for avionics. Their 
first effort resulted in suggested CERs for fire control radars, iner- 
tial navigators, computers, and doppler navigation radars [4]. 


-'Digital Avionics Information System. 
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The CER that was produced for computers in the GRC study 
attempted to take into account the rapid pace of technological change 
in this field by adjusting the data base not only to 1974 dollars, but 
to 1974 technology. The CER produced thus reflects only 1974 technol- 
ogy and is incapable of estimating the costs of future technologies 
without using a suitable technology change factor. With technology 
changing as rapidly as it is in the computer field, estimating this 
factor may be difficult. This point not withstanding, the CER itself 
is based largely on the weight of the computer, but it also includes 
inputs concerning the computational speed, memory, size, etc. 

GRC, in a study for NASA Houston, published in 1975, examined 31 
airborne and spaceborne computers, and developed two CERs, neither of 
which was developed from a data base applicable to the AAS. The first 
was for memory cost, and is stated as 

Ln(memory cost, in dollars per bit) = 8.119 - 0. 149 (year-1900) 

The second concerned central processing units (CPUs) and related their 
costs to the date of development, word length, speed, number of 
bidirectional I/O channels, and whether or not the unit is a space 
application. Both of these CERs were developed from a data base of 
expensive units, the cost of which ranged from $32,000 to $683,000. 
It was judged that this data base was out of the cost range of the 
equipment to be used in the AAS. 

The Air Force is also an indirect sponsor of avionics cost- 
estimating activities through the companies that manufacture its air- 
craft. Lockheed is an example, especially with their series of 
antisubmarine warfare airplanes. However, CERs do not serve the kinds 
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of needs that Lockheed feels it has, and therefore it has neither 
developed nor used them. 

Navy Studies on Avionics 

The Naval Air Development Center (NADC) has been engaged in the 
development of avionics CERs for some time. The Center mounted an 
extensive effort involving avionic computer systems in 1970 and 1971. 
This effort resulted in a cost-by-function model that is based on a 
design-engineered approach. It works by using the input parameters to 
design and assemble a hypothetical computer to handle the functional 
load. Then, from a data bank, the model retrieves data on related 
existing equipment. The data that most closely approximate the system 
under evaluation are used as a baseline for cost extrapolation. This 
system depends on the data base, and the limitations in data have res- 
tricted the utility of the model. In concept, however, the model is 
similar to the RCA PRICE model. 

NADC is continuing its avionics cost work at the present time, 
under joint Navy and NASA funding. Their objective is to determine 
top-level avionics costs for various categories of military aircraft, 
so that they can project future trends and assess probable avionics 
costs for different mission aircraft. They are presently almost fin- 
ished with data collection, and analysis will follow. The tools that 
they intend to use are stated to be: 

Experience and expertise 
Basic theoretical relationships 
Existing CERs 
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PRICE model 

Designer experience and projections 

At the present time there is little that can be used from this 
effort. 


Other Defense Efforts 

The most notable effort has been by RCA Defense Avionics, where 
the PRICE model [18] was developed. PRICE is a proprietary model that 
requires the user to attend a special school prior to its use, and 
payment to RCA for its use. As mentioned above, the PRICE model is 
widely used for the estimation of military avionics costs. 

Much of the model is based on data from RCA's own records and 
includes other firms' equipment. (There are over 6000 items in the 
file.) There are 56 model inputs that are used to describe the equip- 
ment for which costs are to be estimated. The inputs are either com- 
plexity factors or physical characteristics. Some are mandatory, oth- 
ers can be calculated by the model. Some are calculated by the model 
regardless of whether or not they have been provided as an input, 
e.g., engineering performance schedule complexity factor, total com- 
ponent count, etc. This provides a check and balance to the accuracy 
of the system descriptors used as inputs. 

Unfortunately for the novice user, over 75 percent of the inputs 
appear to require the user's judgment with regard to the complexity of 
such things as structure, electronic circuitry, and level of engineer- 
ing technological improvement. The model is apparently so general 
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that there is no requirement to specify the type or general class of 
equipment for which costs are desired. Little is known about the 
basis of PRICE, other than the statement by its developers that it is 
based on a dollar per pound relationship. Without being able to look 
at the data base used and the equations (standards) developed, it is 
impossible to determine the statistical validity of the model. How- 
ever, whether or not the model is statistically sound, results from an 
earlier NASA study, shown in Fig. II-l, indicate that in the hands of 
an experienced operator the model can provide estimates that are close 
to actual realized costs. 

The subject of previous avionics studies can be summarized as 
follows: 

o Most studies of military avionics have not resulted in 
the development of broadly useful parametric CERs. None 
were found that could be used in this study, 
o No previous CER studies of civilian avionics were located, 
o The problems inherent in developing avionics CERS stem 
from the fact that a given function may be produced 
by a variety of electronic methods of widely varying cost. 

This obviates the required relationship between functions 
and costs. The problem is generally manifested by the 
discovery that the published data on the equipment is 
often not related to its cost, and that conversely, the 
cost determinants of the equipment are often unpublished. 

The conclusion of our search for existing tools to assist in the study 
is that they do not exist. What tools are required must be developed. 
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Equipment 

PRICE 

Computer 

Program 

Actuals 

Adjusted 

Actuals 

Net 

Variance 

ATCA 

11,538 

13,096 

f 17 ■ 

12,076 

-4.5% 

DECA 

4,834 

5,114 

4,784 

+1.1% 

RR Radar 

55,966 

61,420 

58,561 

-4.6% 

Transponder 

18,553 

20,244 

19,300 

-4.3% 

Landing Radar 

32,997 

35,776 

33,776 

-2.3% 

Laser Altimeter 

2,430 

2,962 

2,568 

-5.4% 

VHF Transceiver (LM) 

6,570 

6,258 

6,258 

+5.0% 

VHF Transceiver (CSM) 

3,968 

4,091 

4,091 

-3.0% 

VHF Transceiver (LM) 
Ranging Mod. 

2,645 

2,758 

2,748 

-4.1% 

VHF Transceiver (CSM) 
Ranging Mod. 

1,728 

1,768 

1,768 

-2.3% 


SOURCE: Frank Freiman, RCA Avionics Cost/Schedule Study, NAS 9-12356, 1972. 

NOTE: Program Actuals Compared with PRICE Computer Runs — Flight Hardware 

($ 000 ) 


Fig. II-l — Example of estimation precision obtained by RCA PRICE model 
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THE DEVELOPMENT OF CERS 

Review of previous work revealed that significant problems 
existed in the development of avionics CERs. Also, a review of the 
two Preliminary Candidate Advanced Avionics Systems (PCAAS) developed 
for NASA by Southern Illinois University (SIU) and Systems Technology, 
Inc., (STI) [13,14] showed that two extremes were represented. One of 
the PCAAS was designed around existing components available off the 
shelf, and the other was a completely integrated system designed to 
function as a single unit. As explained in the Introduction, a cost- 
estimating method that treated the entire avionics suite as an entity 
was highly desirable. However, data on complete entities do not 
exist, and therefore a historical base from which to start is lacking. 
Further, the exact problem that has been discussed concerning dif- 
ferent ways to achieve the same performance is presented by the two 
PCAAS systems. One resembles a grouping of familiar avionics com- 
ponents that have been hooked together to perform as a unit. The 
other resembles nothing that a general aviation pilot has heretofore 
seen, since it has been designed as a totally new unit. Yet both 
presumably serve the same function. A single cost-estimating method 
cannot accommodate both suites; they are simply too different. 
Because of this, we approached the development of CERs in two ways. 

First we attempted to develop CERs for off-the-shelf equipment, 
with the intention of using these CERs for those PCAAS that could be 
identified as being made up of existing components, in contrast to a 
completely integrated system. Then, for those PCAAS that were 
designed as integral units, we attempted to develop a different set of 


CERs . Both of these efforts will be described in detail below. 
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With two different cost-estimating methods, consistency of esti- 
mates could be expected when like systems were being compared, i.e., 
when one system made up of off-the-shelf components was compared with 
a similar system, or when integrated design systems were compared with 
each other. This type of consistency disappears, though, when systems 
of different design, that each require a different cost method, are 
compared. The analyst must understand this, and must balance the 
problem of having no estimates to compare with the fact that different 
methods probably produce results that are only qualitatively compara- 
tive. 

When one CER is used to estimate the costs of two or more sys- 
tems, the cost differences are consistent from one comparison to 
another because the CER translates the functional inputs into cost 
outputs according to the algorithms built into it. If one CER is used 
to estimate the costs of one piece of equipment and a different CER is 
used to estimate the costs of another, the cost differences observed 
may not be due to differences in functional inputs. This is usually 
caused by the fact that the CERs have different inputs, and the cost 
estimates are then based on different parameters. An example of this 
important point will illustrate it. Suppose that there are two kinds 
of commercially available transmitters, each using a different tech- 
nology. For one kind, a good CER might result from relating cost to 
power output. For the other, the best that can be done might be to 
relate cost to weight. Now suppose that we want to compare the costs 
of the two types based on their power output. Clearly if this is to 
be done, the analyst must be able to translate power output into 
weight for the second kind of transmitter. But if he could do that, 
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he probably could have developed a CER based on power in the first 
place. On the other hand, if all of the transmitters were of the 
first kind, the CER would be the embodiment of the change in cost as a 
function of a change in power. The use of single CERs is more mean- 
ingful in making consistent comparisons. 

The above discussion presumes that the data bases for developing 
both kinds of CERs were equally good. Seldom is this the case. 
Therefore the problems are compounded, even when the different CERs 
are more similar than those mentioned above. Poor data bases produce 
poor CERs; good data bases may produce good ones. Comparisons of 
costs that are estimated by using both good CERs and poor CERs will 
have associated low levels of confidence. 


CERs -- Existing Avionics Equipment 

The previous section mentioned that one PCAAS design already sub- 
mitted to NASA was based on off-the-shelf components. The use of a 
CER to estimate the prices of these components is clearly inappropri- 
ate, because the prices are known exactly. However, CERs based on 
existing equipment can serve an extremely useful purpose. They pro- 
vide the analyst with the tools necessary to perform a sensitivity 
analysis, and to answer questions of how costs may change as some of 
the performance features are altered. They allow the analyst to com- 
pare systems with different characteristics (such as power output) by 
equalizing them through the use of a CER to the same power output. 
They also allow the conceptual design of a system that is composed of 
components that are similar to, but not exactly the same as, off-the- 
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shelf items. Based on this utility, we attempted to develop CERs 
based on commercially available avionics equipment. 

We began the effort by obtaining data from the April issues of 
Business and Commercial Aviation , and from manufacturers' literature. 
We used data for the years 1973, 1975, 1976, and 1977 (1974 data were 
not available in time). These data were segregated by class of equip- 
ment, i.e., transceivers, transponders, etc., and all listed attri^ 
butes were coded and entered into machine storage. Statistical ana- 
lyses of the data were then made by using a multiple regression 
analysis program to test preliminary hypotheses about cost/performance 
relationships . 

Analyses were made of communications transceivers, transponders, 
navigation transceivers, and distance-measuring equipment. The 
results of these analyses were unsatisfactory in several respects, as 
follows: 

1. No sensitivity to the model year of the equipment was 
indicated. Thus, cost trends could not be established, 
and the effects of changing designs, manufacturing 
techniques, and new technologies, which ordinarily 
would be expected to show up as a time trend, were 
unobserved. This might be implied to mean that 
manufacturers hold prices (in current dollars) 
relatively constant, and absorb the effects of infla- 
tion through economies resulting from the use of new 
technologies. Our interviews indicated that this 
process not only took place, but that greater 
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capability was continually being built into the equip- 
ment as well, as a result of changing technology that 
allowed this to be done without increasing prices. 

2. The only explanatory variables that were consistently 
significant in the regression equations were power 

and weight, and these frequently explained only a small 
part of the data variance. There are some limited cases 
in which these two variables would be useful in a cost 
analysis, especially if there were other variables in 
the equation as well. But when one or the other of them 
is the only variable that has significance, the 
regression equation is almost useless. It does an 
analyst little good to know how the price of a 
navigation transceiver varies with its weight, when what 
he really wants to know is how its price varies with its 
performance . 

3. Where equations were developed that included the 
limited variables discussed above, they frequently 
produced results that duplicated the data inputs with 
a margin that was judged to be too wide--plus or 
minus one-third was typical. 

The poor results were attributed to the major problem of coupling 
price data to the listed physical and performance characteristics of 
the equipment. For communication transceivers, we had the following 
numerical data to work with: 



- 27 - 


Year model 

Number of channels 

Power output 

Number of boxes 

Weight 

Price 

Manufacturer 

But there were numerous other features that were available on 
some of the transceivers in the data sample. For example, included as 
listed variables were such features as: 

Remote mounted power amplifier 

Blue/white lighting 

Self test 

TSO categories 

Automatic squelch 

Speaker amplifier 

Storage and recall of additional frequencies 

Electronic frequency readout 

Portable battery pack option 

Dual frequency selector 

Self-contained cabin amplifier 

Automatic voice leveling 

Then there were other variations that were not listed, such as basic 
differences in circuit approaches, etc. Costs were judged to be more 
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a function of attributes of the equipment items that were not listed, 
and which are probably nonnumeric (circuit design is an example), than 
of the descriptors that are available. As a result, the statistical 
analysis of the data was terminated. 

ANALYSIS OF NEW TECHNOLOGIES 

New technologies applicable to the AAS were identified in two 
ways. The more direct means of identification was through examination 
of the two specifications for the PCMS that were submitted by the 
NASA subcontractors. The STI specification used more elements that 
can be described as "of f-the-shelf" items than did the SIU specifica- 
tion. Even so, the individual pieces of equipment were tied together 
by a bus structure and fed into the central processor unit. Both the 
bus and the LSI circuitry of the CPU can be considered as relatively 
new technologies to general aviation, especially on the scale of the 
PCMS. The SIU specification, on the other hand, uses far fewer 
"off-the-shelf" items. It incorporates the bus structure and relies 
heavily on LSI circuitry. In addition, it uses a flat plate area 
plasma display, and has provisions for advanced sensors. 

The second means of identifying new technologies was through the 
manufacturers that were visited. In some cases, the visits were made 
specifically for the purpose of looking at new technologies. In oth- 
ers, the visits were for other purposes, but information about new 
technologies emerged during the discussions. 

All technologies that were identified were considered applicable 
to the MS. In general, the technologies that were considered were 


grouped as follows: 
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LSI and microprocessors 

Displays (mainly flat plate displays) 

Bus structure, with associated interfaces 
Sensors 

Software (this will be the subject of a separate later 
discussion) 

Information about LSI and microprocessor technology was taken from 
current literature, and from studies that have been performed for NASA 
on the subject. This information was supplemented by visits to pro- 
ducers and users of this type of equipment, as well as to avionics 
manufacturers. Intel Corporation, Hewlett-Packard, Delco, and Collins 
Radio are examples of organizations that provided helpful information. 

Displays are the subject of great interest by avionics manufac- 
turers, and a variety of technologies are being pursued. Most of 
these however, are being pursued for use as alphanumeric displays. 
Area displays are much less frequently encountered, and where they 
are, they are under investigation for military use. Most information 
on displays originated from Delco and Hughes, as well as from the 
literature. 

Bus structures are analogous to LSI in many ways. They are in 
use at the present time, their parameters are well known, and they are 
subject to fairly dynamic forces that result from technological 
improvements, manufacturing changes, and market pressures. All of 
these indicate lower future costs. Hewlett Packard, who manufactures 
an IEEE 488 bus, provided most of the information. 
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Sensors were found to be the most promising but least-well- 
defined area regarding technological change with associated cost 
changes. The situation was best described by one person who said that 
the manufacture of sensors was "moving into the IC houses." The prob- 
lem is not simple though, because there is such a diversity of promis- 
ing technologies, and because knowledge of many of these technologies 
is so new and so small that it is difficult to say which are the most 
promising. In aggregate, all indications are that there are many 
changes on the horizon, and that these changes almost universally are 
harbingers of more accurate and lower-cost sensors that are free of 
mechanical maintenance problems. 


LSI / Microprocessors 

The most common method of estimating costs in this fast-moving 
technology is by piece parts count. Attempts to use functional 
characteristics founder on the rate of change in the technology. 
What required an entire printed circuit board yesterday, will require 
only one LSI chip tomorrow. While this concentration of processing 
power is occurring, the cost of circuit boards remains about constant. 
It is for this reason that the piece parts count makes good sense, and 
gives good results. The method has other attractions as well. Piece 
parts are assembled into printed circuit (PC) boards of a given size 
and complexity. PC boards of like size and complexity are found to 
have similar piece parts costs, and therefore similar costs per board. 
It is common for people "in the industry" to talk of PC board prices 
(which are a multiple of costs) of about $1000 per board, whether the 
boards are fpr one specific application or another. It is also common 
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for industry sources to say that the retail price of an item is equal 
to eight times the piece part cost. This multiple takes into account 
labor, overhead, G&A, profit, distribution and sales costs. The exact 
amount of each of these factors may vary from company to company, but 
the results appear to be surprisingly uniform in aggregate. 

PC boards do vary in both size and complexity, and these vari- 
ables can be taken into account in the piece part count, while still 
using the rule of thumb that the finished PC board will sell for eight 
times the piece part cost (e.g., if the piece parts have a cost of 
$100, the PC board will sell for $800). 

These relationships have been used in both the STI and STU 
specifications of the PCAAS. SIU derived a relationship of price = 10 
times piece parts count, which is probably conservative, based on the 
industry criteria stated above, and used this relationship to estimate 
the cost of their PCAAS. Using a multiple of ten versus a multiple of 
eight can be viewed as a means of hedging on the accuracy of the piece 
parts count, which has some inherent uncertainties at the stage of 
design used by SIU. If their final design required 25 percent more 
piece parts, their cost estimate would probably still be valid. 

STI used these relationships in a less rigorous manner, but still 
appropriately. Virtually all of their system was designed to use com- 
mercially available parts, including the PC boards for the central 
processing unit. However, there were a few PC boards that were not 
commercially available, and the price of these boards required estima- 
tion. This estimation was made by comparing the size, complexity, and 
piece parts count of these not commercially available boards with 
those that were commercially available, and then deciding that they 



-32- 


were sufficiently similar so that the prices would be similar. These 
PC boards, interestingly enough, are priced in the $200-$300 range, 
indicating that they are less complex than the boards generally used 
in the avionics industry. It was mentioned above that $1000 per PC 
board was a common number. The actual numbers obtained from each com- 
pany are as follows: 

Company A, MIL SPEC boards $1000 - $1500 

Company B $ 800 - $1000 

Company C $ 600 - $1000 

The relationship between piece parts cost and retail price was given 
as follows: 

Company C - Retail price = 8 x piece parts cost 

Company D - Retail price = 8 x piece parts cost 

Company A - Government price = 4 x piece parts cost, and 

government price = 1/2 retail price 

The above relationships must be qualified in that they do not 
implicitly contain a measure of performance. It was mentioned earlier 
that the performance of LSI circuitry is increasing rapidly, and thus 
a task that requires two or more PC boards today may require one or 
less tomorrow, and the price of the board may be no different. This 
effect is automatically taken into account if the piece parts count is 
used; but if estimates are being made for the future, it is then 
necessary to estimate how the piece parts count will change in the 
intervening period, or how the performance of the PC boards will 
change. Projections of this type are difficult to make because the 
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present rate of change is very high, and the degree to which it will 
continue appears to be very uncertain. Figure II-2 was taken from a 
Honeywell study of computer technology that was performed for NASA. 
This figure indicates these large rates of change, and could be inter- 
preted that there will be price changes of factors of two or more 
between now and 1980. But the figure is a straight-line projection on 
a logarithmic scale, and such extrapolations are often suspect. 
Whether quantitatively correct or not, the indications are uncontested 
that whatever the cost estimate is for a given microcomputer today, it 
will cost less, and probably substantially less, in the 1980s. 

Displays 

The only area displays that are presently in use in aircraft are 
CRT displays, which are used primarily for radar. Their costs are 
well established, and such displays can be considered an off-the-shelf 
item. Flat plate area displays are mostly in the research stage. The 
Honeywell report sums up the status of these displays by stating "none 
of the various flat-plate matrix displays under development are ready 
for application to general aviation." This opinion was shared by the 
equipment manufacturers. Delco, for example, is working with various 
types of alphanumeric flat plate display technologies, and much 
currently available general aviation avionic equipment uses 
alphanumeric displays. No one was found to be working with flat plate 
area displays for use in civilian aircraft, but Hughes Aircraft Com- 
pany is deeply committed to liquid crystal display (LCD) panels, which 
they perceive as having their first use in military aircraft. 
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Fig. II-2 — Projected price/performance ratio for a family 
of Honeywell processors compared with past and 
present devices 


-35- 


Some cost information is available for alphanumeric displays, 
which is presented in Fig. II-3. No cost information is available for 
flat plat area displays suitable for aircraft use. These displays are 
still under development, and are not expected to be used by military 
aircraft until the early 1980’s. Subsequent to this use, they may be 
expected to be available for civilian avionics application. 

Bus Structure 

As mentioned above, busses are in common use, and prices are 
known for both the cables and the interface circuits. These prices 
were stated by one manufacturer to be as follows: 

Cable lengths: 1 meter $60 

2 meters $65 
4 meters $70 

Interface circuits: $150 to $1000 

What is more interesting to speculate about is the future price 
of the interface circuits, since they consist of LSI, MSI, and hybrid 
circuits, which are subject to the same cost reduction forces that 
have been identified above. In this respect, the manufacturer 
hazarded a guess that the retail price of the least expensive inter- 
faces would reach as low as about $50 in the 1980’s, and that their 
manufacturing cost would be well below $10. If the relationship 
described earlier of 8 x piece parts cost = retail price is applied, 
then a $50 retail price implies piece parts costs of about $6, and 
total manufacturing costs would probably be as they described--well 
below $10. Also, they are essentially projecting a retail price 
reduction from $150 to $50 over the next 5 years or so (in the 
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1980's). This projection would appear to be consistent with experi- 
ence with new, solid state electronics circuits. 

Sensors 

The prices of present sensors are well known, and if these sen- 
sors are used in a PCAAS, as was done by STI , there is no problem in 
estimating them. If, on the other hand, advanced sensors are to be 
used, there is presently no cost information available. But our 
interviews made it clear that sensor technology is on the threshold of 
dynamic changes that will probably reduce acquisition costs substan- 
tially, improve quality, and reduce maintenance costs. 

Attitude sensors will probably undergo the most profound changes. 
There are a number of different technologies that offer promise. 
Prominent among them are ring lasers and resonant quartz "bells." In 
the latter, strap-down quartz bells are excited to their resonant fre- 
quency. When the attitude of the bell changes, it causes a displace- 
ment between the standing wave on the bell and the position of the 
bell. This displacement can be measured as a minute capacitance 
change, and reported as the appropriate change in attitude. No moving 
parts are used, and the entire sensor is electronic. Development of 
this device by Delco is in its infancy, but it was suggested that the 
device would reduce gyro costs substantially. Personnel at Collins 
Radio also concurred that by the 1980s, mechanical attitude sensors 
would be a thing of the past, and that whatever nonmechanical system 
(or systems) would be in use would be much less expensive than present 


systems . 
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There are many pressure sensors on an aircraft, and there are 
presently two nonmechanical devices that are likely contenders. One 
is a resonant quartz crystal that pressure acts on to change the 
resonant frequency. A digital frequency counter completes the system. 
Collins Radio feels that this device would work far better than 
mechanical devices, and that it has the potential of eventually cost- 
ing the same. 

Honeywell has a piezoelectric pressure sensor that relies on a 
silicon diaphram with resistors, similar to a strain gage. There is a 
problem with this device, since it is temperature sensitive, and the 
temperature-induced output is substantially larger than the pressure- 
induced output. This problem has resulted in the use of sophisticated 
bridge circuits, plus software "calibration" for the individual unit. 
However, Honeywell feels that this sensor has the potential of being 
very cheap. 

J-Tech, of Cedar Rapids, has developed an airspeed indicator that 
is based on a pipe with a post in it. A vortex forms behind the post, 
and then moves away from it and dies out. The number of vorticies 
formed per unit time is a function of the airspeed. An electro-optic 
vortex counter completes the instrument. 

The sensors used in the STI specification are off-the-shelf 
items, and their prices are known. SIU uses off-the-shelf attitude 
sensors (gyros) but specifies no particular kind of pressure sensors. 
Instead they advocate voltage-controlled oscillator devices for which 
they have allowed $55 per sensor in their estimate. At the moment, 
there are no cost data on advanced sensors. The development of cost- 
estimating relationships for advanced sensors must wait until the 
technologies evolve further, and data become available. Until that 
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time, the only guide to sensor costs is industry's belief that they 
will probably be lower than the cost of today's equipment. 

CONCLUSIONS AND RECOMMENDATIONS 

The investigation into the development of generalized cost- 
estimating methods for general aviation avionics found that: 

o Previous attempts to develop avionics CERs had only 
limited success. 

o Attempts to develop a single methodology were precluded 
because of lack of data and the problem of dealing with 
as yet undefined future technologies. 

o The development of different methodologies for the two 
PCAAS systems already proposed to NASA was unsuccessful 
in the context of the creation of parametric relation- 
ships that expressed costs as a function of physical 
and performance parameters. This failure was due to the 
lack of significant statistical relationships among the 
variables tested. 

o A number of estimating methods, rules of thumb, and 
industry practices were collected which can be of 
assistance in analyzing equipment costs. 

Because parametric relationships could not be developed, esti- 
mates of the acquisition and maintenance cost of advanced avionics 
suites must rely on techniques that are based on piece parts counts. 
The use of these techniques will allow the cost data for one system to 
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be compared with the cost data of other systems. The ability to per- 
form sensitivity analysis will be limited, or nonexistent. 

NASA cannot predict the types of designs that may be submitted, 
or what technologies might be incorporated into the designs. There- 
fore, NASA should require that each submission be accompanied by a 
contractor's prepared estimate of users' purchase price, maintenance 
philosophy, and maintenance cost estimates. These estimates should be 
backed up by a description of the cost-estimating methodology used, 
and a listing of all factors and analogous and corroborative data. 
The description should be sufficiently complete to duplicate and 
evaluate the estimate. Estimates prepared in this way will most 
likely be engineering estimates that rely upon piece parts count, 
piece parts prices, and manufacturing labor cost estimates. 

The inputs to these cost estimates can be reviewed for their rea- 
sonableness and completeness, or, in the case of technologies that are 
not yet commercialized, for evidence that the estimate made is defen- 
sible. Estimates made in this manner can be used to make comparisons 
among the various designs that are submitted, provided it is under- 
stood that the estimating methods used for different levels of design 
will probably have widely differing (and unknown) estimating accuracy. 
Therefore, comparisons that are made should stress general cost ranges 
rather than exact differences among estimates; they should concen- 
trate, wherever possible, on uncertainties in the estimates. These 
uncertainties could be of critical importance where the technology is 
new, or is noncommercial, or where either the piece parts count or 
prices appear to be subject to doubt. For some systems, it may be 
possible for the contractor to submit the estimate in disaggregated 
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form, so that cost comparisons may be made by functional subdivisions, 
i.e., transceivers, transponder, etc. For integrated designs, this 
disaggregation may be impractical, but at least the hardware and 
software may be separated. 

System integration, particularly for systems that are made up of 
off-the-shelf components, is likely to be one of the driving costs, 
and NASA should request that this subject and the subject of software 
be addressed explicitly and in detail. 

Using the above techniques, certain assessment of costs is possi- 
ble. However, the ability to perform some types of analysis will be 
constrained by the fact that these estimates will rely upon engineer- 
ing cost estimates. That will be particularly true in the case of 
sensitivity analysis, where functional relationships are more useful 
than parts counts. As an example, consider two designs, one of which 
is clearly superior to the other, but which has a higher estimated 
initial cost. Suppose that the superior design has a higher tran- 
sceiver output than the other design, and that the additional power is 
judged; unnecessary. A reasonable question to ask is how the cost of 
the better unit might change if the transceiver power were reduced. 
With the right kind of parametric relationship, this exercise would be 
simple. Without such a relationship, the exercise cannot be done 
unless the unit is redesigned and a reestimate is made based on the 
new piece parts count and description. This example serves to illus- 
trate that each estimate that is considered represents one point on a 
curve that describes cost as a function of various parameters. Having 
the functional relationship gives the analyst an understanding of 
cause and effect, and the ability to test the cost sensitivity to 
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changes in the values of the parameters. When it is not possible to 
develop these functions, as for the AAS, simpler comparisons are pos- 
sible, and often sufficient. 
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III . RELIABILITY OF ADVANCED AVIONICS SYSTEMS 

This section will briefly describe the state of the art of relia- 
bility prediction and reliability assessment for avionics equipment. 
It will be shown that there is currently no comprehensive and 
trustworthy methodology for achieving the desired predictive capabil- 
ity discussed in Section I of this report. Furthermore, the lack of 
any significant amount of data on reliability and maintainability of 
current avionics systems used in general aviation makes it impossible 
to develop new prediction methods responsive to the objectives of this 
study. It will be seen, however, that even within these recognized 
and substantial limitations, reliability prediction and assessment can 
be accomplished to a worthwhile degree, particularly in the important 
aspect of comparing either the projected or achieved reliability of 
two or more competing designs, and also in identifying those portions 
of any single design which have a disproportionate likelihood of being 
troublesome . 

This section will address these several facets of reliability, 
including reliability prediction, design reliability, quality control, 
environmental factors, and maintenance (though most aspects of the 
last-mentioned subject are in Section IV). In order to accomplish as 
much as possible in these matters, we have chosen not to restrict our 
sources of information to general aviation per se, but to incorporate 
commercial and Air Force experience wherever it seems appropriate. In 
the case of the Air Force experience, some specific reasons for 
including this information will be given later. 
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WHAT IS RELIABILITY ? 

Reliability has been defined as the probability of failure-free 
operation within the design performance limits for a given period of 
time under specified operating conditions. Reliability is usually 
expressed as a decimal fraction or as a percentage of uptime to total 
time (this latter quantity should more correctly be labeled availabil- 
ity). Conversely, unreliability is the probability of a failure dur- 
ing a given period of time. Many times it is useful to express the 
reliability of a device in terms of mean time between failures (MTBF), 
which is the average hours of operation between failures as measured 
on a sample group of units for an extended period of time. It should 
be emphasized that MTBF and reliability are not synonymous terms. 
However, a precise mathematical relationship does exist between them. 
For a fair variety of situations, including most involving electronic 
equipment, a reliability equation can be written which illustrates the 
most general parameter relationships: 


t 

m , 

R = e 

where R is the reliability, less than or equal to 1.0, 

t is the required time of operation, usually hours, and 
m is the mean time between failures. 

The equation shows that only two routes to improved reliability 
are possible: reduce the operating time, or increase the MTBF. Since 
the former is usually not possible, the importance of the latter is 
made clear. This and other terms for quantifying reliability and the 
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relationships between them are discussed in detail later in this sec- 
tion. 

Although reliability engineering as a discipline has been esta- 
blished for over 20 years, there are still many areas that are not 
well understood, and many problem areas lack formal approaches to 
solution. The major areas of reliability science are reliability 
prediction, design for reliability, reliability measurement, and reli- 
ability improvement. The most common misconception in this regard is 
that a given level of reliability is built into a piece of equipment 
when it leaves the factory, and that built-in reliability is what the 
user, on the average, can expect. However, this is not at all what 
happens. Reliability depends not only on design, but also on quality 
control during manufacture, parts screening prior to installation, 
failure mode analysis, quality of maintenance, user training, environ- 
ment, scheduled maintenance, etc. 

SURVEY OF TIE DEVELOPMENT OF RELIABILITY PREDICTION METHODS 

During the early days of reliability prediction efforts, it was 
universally recognized that the reliability of a device was severely 
affected by its complexity, giving rise to the admonition to "keep it 
simple." Thus, the early substantive efforts concentrated on a meas- 
urement of the complexity of a device, for which the active element 
group (AEG) became the usable proxy in electronics. This type of 
prediction is exemplified in the so-called Bird diagram,* which gives 
a failure rate per operating hour as a function of the analog func- 


*George T. Bird, ARINC Research Corporation. 
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tional complexity [Fig. III-l ] . In addition, different relationships 
are associated with different environmental operating conditions, the 
least severe being storage, whether in depot or on ship, and the most 
stressful being the missile flight environment. For the less stress- 
ful of these environments, the failure rate was believed to be linear 
with complexity (slope = 45 degrees). For the more stressful environ- 
ments, the failure rate depended on the complexity to a power greater 
than 1 (slope > 45 degrees). 

The Bird diagram was satisfactory during the days when electron- 
ics equipment consisted of vacuum tube circuits. Since nearly all 
vacuum tubes had an electrically heated cathode and other conducting 
elements, their failure characteristics in circuits were reasonably 
alike. There was a known degradation phenomenon associated with the 
heated cathode emitter, which was certain to make itself felt at some 
stage in the life of the device. When transistors came into wide use 
in electronics, the usefulness of the Bird diagram declined, since 
transistors did not have the same mechanism of degradation associated 
with the heated filament in a vacuum tube. Furthermore, the Bird 
diagram implied that no significant improvement in reliability could 
be made through time, without a decrease in the complexity of the dev- 
ice. During the 1960s, it was clear that improvements in the relia- 
bility of certain high-quality equipment were taking it out of the 
realm of the Bird diagram prediction. 

At this stage, it became necessary to account not only for the 
vacuum tube, which had been the dominant contributor to unreliability 
of the active element group in the tube-type equipment, but rather to 
account for all the piece parts in a given electronic circuit. Moving 
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Equivalent Analog Functional Complexity, N Active Elements 
Fig. III-l — Bird diagram 
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in this direction, the most capable researchers in this field 
developed failure rates for individual piece parts; these rates were 
to be tabulated during the design phase of a device and then added up 
to provide an estimate of the failure rate of the assembly. One focus 
of this activity was at the Air Force’s Rome Air Development Center, 
which developed the first handbook for estimating electronic reliabil- 
ity. This publication, MIL Handbook 217, was published in the early 
1960s. It tabulated hundreds, if not thousands, of failure rates for 
individual piece parts, and had a few reliability failure models for 
different types of equipment. 

The first version of the handbook was followed by two others in 
1965 and in 1974. The latter, MIL Handbook 217B [77], contained much 
larger tabulations of part failure rates, including derating parame- 
ters, environmental stress parameters, and a larger number of relia- 
bility failure models. Included in the tabulation were the first 
results for modern electronic devices such as monolithic circuits, and 
bipolar digital devices, bipolar and MOS linear devices. 

MIL Handbook 217B has had two updates. The first added microwave 
transistors, high-power tubes, lasers, and cooling blowers and fans to 
the original tabulations. The second update, originally scheduled to 
be completed late in 1977, will add opto-electronic semiconductors, 
including light emitting diodes (LEDs), displays of LEDs, and optical 
isolators. It will also include further information on high-power 
microwave tubes and some new information on failure models for new 
hybrid integrated circuits. 

It should be noted that the second update involves devices which 
are already in wide use, both in military and commercial systems. 
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This means that MIL Handbook 217B can be considered to be at least 1 
and probably 2 years behind the technology of these devices and their 
wide application. Inevitably, any reliability prediction methodology 
based on failure rates of individual piece parts must lag the technol- 
ogy it covers, at least until considerable experience has been gained 
with the type of circuitry in question. Therefore, one shortcoming of 
any reliability prediction methodology must be expected to be the lag 
behind current technology. 

Mil Handbook 217B contrasts in another way with the Bird diagram. 
The earlier method was directly usable in the conceptual phase of sys- 
tem design, since so little detail was required to apply it. The more 
recent method requires detail design knowledge in order to account for 
all piece parts. Thus, one useful facet of a prediction method was 
lost in the transition. 

The most recent significant development in the field of elec- 
tronic reliability prediction concerns computer-aided evaluations of 
reliability. Some years ago, Kenneth Blemel, formerly of R/M Systems 
in Albuquerque, New Mexico, developed a computer reliability predic- 
tion technique for the U.S. Air Force. This method is currently being 
used by the Air Force’s Logistics Command to estimate spare parts 
demand during electronic system lifetimes. The method is primarily 
aimed at logistics considerations, and has limited usefulness for 
direct reliability predictions [62]. 

Another computer prediction model was built by Gaertner Associ- 
ates for the U.S. Army [81]. Although not directly usable for elec- 
tronics reliability prediction outside of military specification 
parts, this method was adopted by Rome Air Development Center as the 
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foundation for a prediction model to be used by Air Force electronics 
development sources. Once again, the method uses military standard 
parts nomenclature in evaluating likely reliabilities. 

The most recent effort in this field, which has a much better 
chance of being used in the commercial environment, occurred at the 
Naval Weapons Center, Ridgecrest, California, in collaboration with 
Systems Consultants Incorporated, also of Ridgecrest. This Navy- 
sponsored development uses commercial descriptions of piece parts for 
input parameters to an evaluation model. The model, called 2I7B 
Predict, uses roughly the methodology of the 217B handbook, without 
requiring the MIL standard designation of parts [75] . In addition, 
the characteristics of standard commercial parts can be input by the 
user so that a complete reliability prediction can be made without 
requiring a relationship to published 217B reliability figures. 

The objective of these computer evaluation models has clearly 
been to reduce the amount of effort and time required to evaluate the 
likely reliability of a given electronic design. It is probable, how- 
ever, that this effort will have some additional benefits, involving 
the elimination of errors of omission or commission which are very 
frequent in this type of evaluation. The computer is unlikely to 
overlook individual elements in a circuit, or to fail to require the 
input of certain environmental parameters. Computer evaluation should 
improve the inclusiveness of reliability predictions, if not other 
aspects of their quality. 
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THE ACCURACY OF RELIABILITY PREDICTIONS 

A key element in an understanding of the state of art of relia- 
bility prediction methods today is the fact that such methods yield 
very poor prediction precision. A review of how good such predictions 
have been over the past 25 years will set the stage for the subsequent 
discussion of current analysis methods. This review will, of neces- 
sity, draw almost entirely on military experience. 

About halfway through that 25 year interval, researchers from 
Rome Air Development Center reported on the quality of reliability 
predictions made during the first 10 years of their intensive 
activity. Table III-l shows a comparison of gross field MTBF measure- 
ments with predictions for several ground electronic equipments [82]. 


Table III-l 

GROSS FIELD MTBF MEASUREMENTS (NORMALIZED TO PREDICTED VALUE) 


90% Confidence 


Predicted 


Field 

Interval 

MTBF 


Point 

Lower 

Upper 

(Hrs) 

Equipment 

Estimate 

Limit 

Limit 

320 

Computer 

1.09 

.99 

1.21 

850 

I/O Controller 

2.31 

1.64 

3.35 

490 

Memory Element 

1.35 

1.21 

1.51 

580 

Message Processor 

2.11 

1.61 

2.80 

835 

Drum Controller 

1.33 

1.12 

1.60 


Radar (1) 

1.27 

1.12 

1.45 


Radar (2) 

1.80 

1.54 

2.15 

513 

Data Display Console 

0.097 

0.094 

0.099 

4131 

Status Display Console 

0.25 

0.23 

0.27 

179 

425-L Display Console 

0.88 

0.79 

0.99 

65.2 

473-L Display Console 

1.03 

(Not Ava 

ilable) 


The agreement between prediction and field results is relatively 
good in this comparison. However, one would have to read the original 
report carefully to note that the equipment being tested "in the 
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field" was in air conditioned laboratory environments. Furthermore, 
numerous corrections to observed data had been made to eliminate cer- 
tain failures which were deemed not appropriate to count against the 
inherent equipment reliability. The table shows that experience data 
reflects, with reasonable faithfulness, the predictions made for the 
equipment in question. Unfortunately, this tabulation of achieved 
reliability relative to predicted reliability is not at all typical of 
most prediction experience, particularly for airborne equipment. 
Indeed, a report by RADC researchers 1 year later showed that achieved 
reliability of a larger sample of equipments ranged downward to one- 
fifth of predictions [63]. 

Needless to say, some care is essential to the evaluation of such 
data. One can infer directly from the Bird diagram that from a relia- 
bility point of view the laboratory is most benign, with other 
ground-based environments somewhat more hazardous, and airborne vehi- 
cles still more stressful. Thus, evaluation in a laboratory, while 
probably necessary at one stage of development, must not be inter- 
preted as representative of end use. By the same token, testing and 
maintenance by the manufacturers skilled technicians may not ade- 
quately represent treatment by owners or fixed base operator (FBO) 
maintenance mechanics. What counts is reliability in the hands of the 
user, warts and all. 

In line with that notion, a more representative tabulation of 
achieved versus predicted reliabilities was the subject of a response 
from the Comptroller General of the United States to an inquiry from 
the United States Senate a few years ago.- The following table shows a 

-The letter, from Elmer B. Staats to Senator Gravel, was printed 
in the Congressional Record for December 9, 1974, pp. S20775-S20776 . 
This letter has been widely used and misused, as in the Reactor Safety 
Study, Wash-1400 (NUREG-75/014) , Nuclear Regulatory Commission, Appen- 
dix XI, pp. 3-15 to 3-21. 
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comparison of aircraft radar subsystem reliabilities achieved in 
operation compared with their specified MTBFs which were the result of 
reliability predictions. Typically, the achieved reliability was 
one-fourth to one-third of that specified. In one case, the achieved 
reliability was less than 1 percent of the specified MTBF. 

Specified Achieved 
Aircraft MTBF- MTBF- 


F-4B 

10 

4 

A-6A 

75 

8 

F-4C 

10 

9 

F-lll A/E 

140 

35 

F-4D 

10 

10 

A- 7 A/B 

90 

30 

A- 7 D/E 

250 

12 

F-4E 

18 

10 

F-111D 

193 

less than 1 

F-4J 

20 

5 


-Approximate figures . 

One paragraph from the Comptroller's letter is particularly 
worth noting: 

NASA experts believed that "absolute" reliability 
numbers are misleading and that the time required to 
develop them is better spent on critical-component 
reliability analyses. [NASA] does make predictions 
during development to compare design alternatives and 
to evaluate components. 

A somewhat more recent examination of the quality of the relia- 
bility estimates was done by Hughes Aircraft Company for the Rome Air 
Development Center. That report, titled "Operational Influences on 
Reliability," found, for example, that the field reliability of a par- 
ticular piece of avionic equipment depends heavily on the type of air- 
craft in which it is used [64] . A more pertinent observation for our 
purposes is that field reliability is significantly less than 
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contractor-predicted values. In this connection, the contractor- 
predicted reliability of the equipment surveyed, when compared with 
the reassessed relevant failure rate, showed a ratio of nearly three 
to one. The researchers concluded that contractor estimates exhibited 
"some degree of optimism." Also noted was the fact that equipment 
used on long-duration flights showed significantly higher reliability, 
in terms of MTBF, than similar equipment used on short flights, a 
phenomenon which we will examine again somewhat later. 

Perhaps the most significant finding of the Hughes study is that 
field reliability ranges from one-third to one-eighth of the predicted 
reliability for the equipment studied, depending mainly on how one 
defines a failure. The Hughes study found that of the failures which 
were deemed to be "relevant," 45 percent were due to operational fac- 
tors and 55 percent to environmental and other factors. These, in 
turn, were outweighed by deficiencies that were attributed to defini- 
tional factors, which constituted about four times as many "failures" 
as those remaining when the so-called non-relevant ones had been 
removed. 

Attempts have been made to develop methods for estimating relia- 
bility in the preliminary design phase, when good information is not 
yet available concerning exactly how many piece parts will actually be 
used and other features of the design that involve such things as 
derating, redundancy, etc. A study of such a method was done by 
Hughes Aircraft Company in 1974 for Rome Air Development Center.* 


*Hughes Aircraft Company, "Study of Reliability Prediction Tech- 
niques for Conceptual Phases of Development," DDC No. A001919. 
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Hughes proposed models for both parts count and failure rate or relia- 
bility according to a regression study of the data from a number of 
avionics systems. The reader should note the strong similarity to CER 
procedures and results. For the failure rate of communications avion- 
ics equipment, Hughes proposed the following equation: 

Ln(F) = 879.816 - .4436DY + .0029476TL + .04224RL 

+ . 1364PP - . 0004324RBP (RBW) - .0002051TL(PP) 

where F is the failure rate per million hours, DY is the design year, 
TL is the transmitter level in watts, RL is the receiver level in dB 
below 1 mW, PP is the prime power in kilowatts, RBP is the receiver 
band pass and RBW is the receiver bandwidth. The fallacy of this 
equation is that it implies an annual growth of 56 percent in the 
reliability of the device, something which we know from experience 
simply does not happen. A similar examination was made for the equa- 
tion proposed for parts count, and this was also found to have a 
tremendous annual improvement factor, which is not reflected in 
current technology of avionics. The similarity between this modeling 
failure and CER shortcomings described in a previous section are real 
and significant. 

Where do reliability predictions go wrong? Deterioration from 
otherwise justifiable predictions of reliability of electronic equip- 
ment can occur in two general areas: the first area is in the factory 
environment where the manufacturer of the equipment has control of it; 
the so-called, in-house region. The second place where reliability 
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deterioration can occur is in the field or in customer use. A brief 
discussion of this type of reliability deterioration will be given in 
a later subsection. 

The manufacturers responsibility is further broken down into 
design, manufacturing or vendor control, and workmanship topics. The 
effect of design on reliability has been studied intensively. Each 
avionics manufacturer typically has his own set of rules which his 
designers must follow in designing avionics equipment, such as rules 
for derating, parts utilization, design review, etc. Great care is 
also required in the manufacturing environment, as for example, in the 
control of vendor parts which are assembled into an equipment. Mass 
screening techniques have been developed along with burn-in pro- 
cedures, testing procedures, etc., all aimed at catching existing or 
incipient defects before a piece part is assembled into an end item. 
However, another area demands attention: the workmanship of assemblers 
in the factory. Once again, each manufacturer has his own set of 
rules to be followed by the assembly personnel in the factory: rules 
for handling, soldering, testing, etc. 

One factor degrading the inherent reliability of a device is 
deficiencies in the design of that device. Permissible tolerances, 
derating of components, and redundancy in signal paths all can have an 
effect on the end reliability of the device. 

The importance of design inadequacies in failure to meet relia- 
bility expectations is usually underestimated and sometimes overlooked 
altogether. An intuitively appealing explanation is that a designer 
will find it impossible to believe that his creation will behave in a 
manner other than the one he had intended.- In spite of these high 

-See Squires, Frank H. , "Safety Engineering and the Murphy Ef- 
fect," Quality , April 1978, p. 57. 
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expectations, many failures are traced to design defects. A fairly 
recent example involves about seven million passenger cars recalled 
during 1973 for potential safety defects. In more than 200 different 
recall programs, the largest single cause was improper design, not 
random failure. When all inherent causes were summed, design inade- 
quacies of various types accounted for nearly half of all the recalls 
[65,66]. This phenomenon is by no means limited to low technology 
applications. Many years ago, it was found that a large fraction of 
the test failures (actually, malfunctions) in the Atlas missile pro- 
gram did not involve real failures of any kind, but were inherent in 
the design of the missile or its support equipment [67]. 

During the process of manufacture, other factors enter the equa- 
tion. TSO* specifications are not a guarantee of end reliability. 
All they require is that certain specified procedures be followed dur- 
ing the design and manufacturing process. The component quality level 
also has a demonstrable effect on end reliability. Commercial part 
reliabilities are not as good as the reliabilities of "controlled" 
parts, which are seldom used in general aviation avionics. The last 
step in the factory process involves inspection of the product before 
it is shipped, a topic we will treat in some detail. 

One of the more important contributors to an increase of relia- 
bility of given equipment over the years is the manufacturer's inspec- 
tion procedures within his own plant. As an example of how such pro- 
grams work, consider the quality control program for preshipment pur- 


*TSO (Technical Standards Order). These requirements are aimed 
primarily at performance and accuracy objectives. 
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poses conducted by one avionics manufacturer. The old production test 
program, applied to all units before shipment , consists of 48 hours of 
operation at 70 degrees Centigrade, following which a test is per- 
formed on the unit at ambient temperature. Under these circumstances, 
30 percent of the units fail, and are recycled for repair. . In order 
to improve the quality of goods shipped, the manufacturer is testing a 
variety of modified final inspections. The first of them substitutes 
a test at 70 degrees Centigrade for the current test at ambient tem- 
perature following the 48-hour run. Under these circumstances, it is 
found that 45 percent of the units instead of 30 percent will fail 
this test and therefore get recycled. A second candidate of this 
nature is to test at -46 degrees Centigrade. Results are not yet 
available for this modification. 

An even more dramatic change in final inspection procedures 
involves a sampling program in which some units to be shipped are 
operated for 248 hours at ambient temperature in a pre-burn-in 
environment. This operation is then followed by a burn-in process 
involving both thermal cycling and vibration for a period of 248 addi- 
tional hours. In this type of operation, it has been found that 30 
percent of all the failures experienced in the 248 hours of cycling 
are experienced in the first cycle of the process. This means that 
the first thermal cycle has a better ability to ferret out failures in 
the units than do subsequent cycles.* 


*For some additional examples of this phenomenon, see William W. 
Provett, Jr., and Richard S. Ullman, 'Effective Reliability Planning 
and Implementation , M Proceedings, 1976 Annual Reliability and Maintai- 
nability Symposium, IEEE Cat. No. 76 CH0-1044-7RQC . 
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The units tested by all four of these procedures are being fol- 
lowed in the field for a period of 1 year to determine which of the 
four methods is the most cost effective in achieving a desired 
improvement in field reliability of these units. 

A number of factors are now believed to contribute to the typical 
overestimation of reliability which characterizes reliability predic- 
tion. One of the strongest contributors is the pressure to win a con- 
tract. Obviously, a high reliability estimate tends to indicate that 
a particular piece of equipment may be superior to other candidates 
for a particular application.* Another contributor is known to be 
non-electronic portions of otherwise electronic systems. The relia- 
bility of mechanical systems is usually significantly lower than that 
of modern electronics, and considerable efforts are underway to sub- 
stitute electronic for mechanical components of avionics systems. 
Another factor is believed to be the fact that state-of-the-art tech- 

o 

nology, that is, technology which is at the forefront of research, is 
typically used to gain a sales advantage, even in the commercial 
avionics market. Still another factor is the uncontrollable environ- 
ment which may be encountered in aircraft. Operating temperatures may 
not meet the standard of 40 degrees Centigrade which is typical of the 
specified operating environment for a fair number of avionics systems. 
Of somewhat lesser importance is the fact that a particular electron- 


*The previously referenced Comptroller's letter to Senator Gravel 
quotes an unidentified Air Force Source thus: "...where a manufacturer 
is interested in having his equipment look good he can, and will, 
select some of the more optimistic data he can find or generate, to 
use in his reliability predictions. Thus reliability predictions, for 
several reasons, tend to be generally optimistic by a factor of two to 
six, but sometimes for substantially greater factors." 
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ics manufacturer may have no prior experience in building the type of 
equipment which is to be designed for a particular application. 
Finally, a number of manufacturers apparently enter the design and 
manufacturing stages with no reliability plan, a procedure which is 
virtually guaranteed to cause consternation later [68]. 

Given that the first demonstration of the reliability of an elec- 
tronic device is likely to be disappointing, what can be done about 
it? The two favored approaches are to make appropriate design changes 
to enhance reliability, and to eliminate early or incipient failures 
due either to design or manufacturing-caused weaknesses. While other 
approaches are also used, these two procedures seem to show the most 
progress in the improvement of reliability through the early life of a 
system. A General Electric study of several years ago showed an 
interesting relationship between the achieved level of reliability and 
the effort expended in the improvement and design of manufacturing 
techniques for a fair variety of both electronic and mechanical dev- 
ices : 

a 

Cumulative MTBF = kh , 

where h is the total operating hour experience, k is indicative of the 
initial (first hour) MTBF, and a is a growth descriptive parameter 
less than 1. 

Examination of a number of studies which used this so-called 
Duane model sheds some light on how quickly reliability can be 
expected to grow in a development program. The following table shows 
the exponents of the Duane growth model as a function of the reliabil- 
ity effort for three studies, the first by General Electric [69] (the 
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original Duane study) in 1970, Hughes Aircraft [70] (for RADC) in 
1975, and studies by the B-l system program office (SPO) in 1976. 

Table III-2 

GROWTH RATE DATA 


Source Reliability Effort 



Low 

Medium 

High 

General Electric (1970) 

0.1 

— 

0.6 

Hughes Aircraft (1975) 

0.3 

0.37 

0.45 

B-l SPO (1976-1977) 

0.1 

0.3 

0.5 


Each of these sources shows that the slope (i.e., the exponent a) of 
the reliability growth curve can be expected to be low when reliabil- 
ity effort expended is also low, and high when reliability effort is 
intensive. Typical values for low reliability growth are an exponent 
of . 1, while for a high growth the exponent is between .45 and .6. 

In-house measures, which are applicable to both original design 
and reliability improvement efforts, are known to have an impact on 
reliability. For example, RCA Avionics has 20 rules to be observed in 
the design and manufacture of their avionics products. One category 
of these consists of design rules, such as a requirement for minimum 
derating in the use of particular types of parts. Another group is 
concerned with design practices, such as the use of metal film instead 
of carbon resistors in the design. A third category is concerned with 
design review; that is, consultation with engineers who are not 
involved with the particular design to take advantage of their 
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knowledge and experience. Finally, design proof is used in the form 
of a testing program to verify that certain design features are indeed 
as good as they are hoped to be. 

An example of a particular avionics system developed for the U.S. 
Air Force (a horizontal situation display, involving complex elec- 
tronic and mechanical functions) is illustrative of a number of the 
problems and procedures associated with reliability prediction and 
reliability improvement efforts. The candidate system for which ini- 
tial requirements were specified in 1966 had an original MTBF require- 
ment of 580 flying hours. In 1970, this requirement was revised down- 
ward by nearly 50 percent, to 300 flying hours, when the first bread- 
board systems were available. Needless to say, this was a complicated 
system, requiring considerable design effort and ingenuity. The first 
prototypes, tested during 1969 and 1970, actually exceeded the 
required 300 hours MTBF, giving an average of 360 hours MTBF. How- 
ever, when production items became available in 1971, the earliest 
test results showed that only 7 hours MTBF was being achieved. A 
large product improvement effort was begun; it was projected to 
achieve an average of 50 hours MTBF during its accomplishment and to 
result in 125 hours MTBF at its conclusion. While the final results 
for this system are not available because of security restrictions, it 
could be shown, through the use of the Duane model, that the reliabil- 
ity improvement programmed into the effort was of a very high order, 
requiring considerable effort and expense. 

One of the less ambiguous things about avionics reliability is 
that it is growing. Figure III-2 is an example of the type of evi- 
dence available for commercial aviation equipment. There also seems 




Fig. 111-2 — Reliability trend — 727 flight control system 
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to be an understanding among the reliability oriented professionals 
that when an acceptable level has been reached or surpassed, effort 
will be turned in the direction of gaining increased performance, even 
at the cost of a portion of the achieved reliability. 

A seemingly opposite extreme is exemplified by the Air Force, 
which has some reliability growth, but mostly at a much lower level of 
1 to 2 percent per year. To be sure, the Air Force occasionally makes 
significant breakthroughs in reliability improvement. The reliability 
of a redesigned TACAN unit for a new Air Force aircraft increased by a 
factor of approximately 10 to 1 over the old unit, even while costs 
were decreasing significantly. However, this doesn't always happen. 
The reliability of a redesigned UHF receiver-transmitter procured by 
the Air Force in fairly recent years actually decreased while the pro- 
ject cost increased, although the redesign involved only a translation 
from vacuum tube technology to solid-state technology. Part of the 
Air Force's problem (not shared significantly by general aviation) is 
that their equipment is becoming progressively more complex, heavy, 
and expensive. A Grumman study showed that MTBF per thousand parts in 
Air Force avionics equipment actually increased by 10 to 15 percent 
per year from 1960 to 1970 (remarkably similar to the commercial 
equipment) , but that this improvement was almost completely offset by 
equally dramatic increases in complexity (and thus cost) during the 
same time period [71], resulting in the apparent low growth cited. 

For a variety of reasons, we believe that Air Force reliability 
data are relevant to the general aviation avionics reliability pic- 
ture, and not just from a reliability growth point of view. For one 
thing, the Air Force operates in a less than optimal environment as 
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far as temperatures, weather, etc., are concerned. The Air Force also 
has low usage hours, typically 1 hour a day or less for most of the 
aircraft in the inventory. This is far closer to the general aviation 
situation than is the commercial airline experience of 10 or 12 hours 
a day. The Air Force, as just mentioned, experiences slow reliability 
progress, probably more akin to general aviation than commercial 
experience would be. The Air Force also has a considerable variabil- 
ity in the available maintenance skills and practices, due to a rapid 
turnover in the work force. Finally, the Air Force is deeply involved 
in work with digital multiplexed (integrated) avionics systems like 
DAIS, the F-15, the F-16, the former B-l, and the E-3A. These activi- 
ties must eventually generate experience useful in general aviation 
integrated systems. 

The general aviation community seems to fall between these two 
apparent extremes of rapid growth in commercial aviation and slow 
growth in the Air Force. The relatively few references we have been 
able to find indicate that the reliability of general aviation avion- 
ics is still growing, but at a somewhat slower pace than for commer- 
cial aviation avionics. In the case of general aviation, the growth 
seems to be between 5 and 10 percent per year for the same or similar 
equipments [72,73,55,56], compared with the apparent 25 percent or 
more exemplified in Fig. I I 1—2 . 

One possible significance of this observation is that the consu- 
mer is becoming progressively better informed regarding his own self- 
interest in the achieved reliability of specific devices, and he is 
willing to forego increased reliability only if additional valued 
functions are substituted for the improved reliability which would 
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otherwise be expected. The practical impact of this observation is 
that the consumer will seldom accept a system, however impressive its 
performance, if he believes it to be unsupportable from a reliability 
and maintainability point of view. 

RELIABILITY ANALYSIS 

This discussion is concerned with what we know about current 
avionics reliability. Reliability is an elusive characteristic in 
avionics. It’s hard to quantify, for a variety of reasons. The peo- 
ple whom we would expect to be the best informed, the manufacturers of 
avionics, tend in the main to have poor feedback from the field. The 
big users of avionics seem to be the best source of information con- 
cerning its reliability, and the character of those users seriously 
affects such data. Another problem concerns the fact that the usual 
criterion for reliability, the mean time between failure, is an inap- 
propriate measure for many aviation purposes. 

Any assessment of reliability must logically proceed from experi- 
ence data. Compared with the amount of such data needed to develop a 
predictive model by conventional methods, data applicable to the 
owner-pilot portion of general aviation is essentially nonexistent. 
To be sure, attempts have been made to develop experience data, most 
notably by AOPA and AEA, but all such efforts have fallen short of 
even the most minimal requirements for reliability projection. One of 
very few successful efforts on our part to get such information came 
in a decidedly unconventional form, which does seem to have some prom- 
ise, and will be discussed immediately before the Conclusions and 


Recommendations subsection . 
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Most people, when asked where the best reliability information 
might be in avionics, will suggest the manufacturer of the equipment. 
But warranty failure data are notoriously unreliable. The manufac- 
turer has relatively poor feedback from field users. For example, a 
number of unfranchised (usually untried, one-of-a-kind) installations 
of this equipment are made in the field, and their results are seldom 
reported to the manufacturer. Another source of confusion is the fact 
that original equipment manufacturers (OEM’s) of aircraft will fre- 
quently exchange avionics units at the factory when they are being 
installed and will return them to the manufacturer as field failures 
instead of factory failures or vice versa. The manufacturer will not 
usually know how long one of his units sat on the shelf until it was 
either sold by a fixed base operator (FBO) or installed by an OEM. In 
any warranty summary, heavy users of equipment will be over- 
represented, and this is important because their maintenance tends to 
be better than that available to the individual aircraft owner. 
Finally, a major contributor to the poor quality of warranty informa- 
tion is the fact that the number of hours that the equipment is 
operated is unknown. The manufacturer customarily assumes that the 
equipment is used 200 hours per year in general aviation. All of 
these factors conspire to make warranty data very poor for the purpose 
of determining inherent avionics reliability. 

Even if one could get good information about field failures and 
field operating hours, the extraction of mean time between failures 
from these data might still be inappropriate for some purposes. For 
example, an emergency locator transmitter (ELT) , similar to the one 
proposed in the STI avionics suite, shows a purported MTBF of 4500 
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hours according to warranty data.* Meanwhile, it is commonly believed 
that the reliability of these emergency locator transmitters approxi- 
mates only 0.5, a figure based on accident statistics. The low relia- 
bility in service is caused not by a short MTBF, but rather by 
inherent design inadequacies: the fact that a crash is either not 
serious enough to set off the g sensor, or is so severe that the unit 
is destroyed, and therefore cannot transmit. Neither of these 
phenomena come into play in the calculation of MTBF from warranty 
data, nor from any other source. 

The mean time between failure may also be an inappropriate meas- 
ure when considering flight safety. It has been observed that most 
failures occur (or at least are discovered) at the turn-on of an elec- 
tronic device. Once operating, an electronic device is much more 
likely to work during an entire flight than an MTBF specification 
would indicate. This means that an MTBF correctly translated to the 
reliability of any particular length of flight will give a much too 
pessimistic prediction of the likelihood of failure, particularly for 
items with a relatively low MTBF. 

An even more fundamental problem with reliability assessment of 
avionics is that the inherent reliability is seldom achieved, cus- 
tomarily being degraded by a variety of factors. 

The end use of an item is probably the biggest contributor to 
variability in achieved reliability. The environment in which the 
device is used has a substantial effect on the achieved reliability. 


*The most common failure mode is a weak battery. Since the dev- 
ice is normally push-button tested prior to flight, this mode of 
failure is probably not relevant to reliability in use. 
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It is well known that higher operating temperatures result in lower 
reliability. Another factor contributing to the observed reliability 
is the amount of use. Heavy users, as mentioned earlier, get more 
hours between failures from comparable equipment than those who use it 
a few hours a month. The length of a flight has an unexpected effect 
on reliability. Longer flights result in higher observed MTBF, other 
things being equal. This is verification of the phenomenon usually 
called cycling stress, or turn-on stress, which has been demonstrated 
in electronics equipment for more than 25 years. Quality of mainte- 
nance is also a contributor to or a detractor from reliability. A 
demonstration of this will be given later. Finally, other factors are 
known to have an effect on observed reliability in the field, and some 
of these will be brought out in the later discussion. 

All avionics manufacturers seem to agree that the quality of 
maintenance in the field environment has a significant effect on the 
observed reliability of their equipment. As an example of this 
phenomenon, consider the information in the following table showing 
removals and failures, with resultant MTBF calculations, for a VHF 
receiver-transmitter in a cargo airline operation. This receiver- 

transmitter, which is widely used in commercial and even in general 

' ! 

aviation, had shown a fairly steady reliability in this application 
over a period somewhat longer than 1 year. During October, November, 
and December of 1976, however, a significant decrease in the observed 
number of verified failures was noticed. At the same time, the number 
of removals seemed normal. Apparent MTBF jumped by a factor of at 
least two. This was followed by a period of 3 months in the beginning 
of 1977 when removals approximately doubled and failures were verified 
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in nearly all of those removal cases. MTBF dropped to half its normal 
level . 



REMOVALS 

FAILURES 

MTBF 

:0R 14 MONTHS 

188 

149 

1250 

JULY 1976 

11 

11 


AUG 

9 

5 

1591 

SEP 

21 

13 


OCT 

14 

3 


NOV 

16 

7 

3142 

DEC 

21 

8 


JAN 1977 

40 

34 


FEB 

22 

22 

742 

MAR 

16 

9 



What had happened, and was vaguely known by the people involved, 
was that there was heavy pressure during October, November, and 
December of 1976 to get aircraft back into the air when failures had 
occurred. This resulted in poor maintenance of these communications 
transceivers, with the result that they later failed again. But even 
more information is available on this particular situation, which 
attracted considerable attention at the organization involved. The 
number of removals per aircraft was tabulated for the first quarter of 
1977. The expected value was 2.4375. Seven or more removals were 
observed in four aircraft, which is far above the expectation for this 
number of removals. This is an indicator that something might be 
wrong on those particular aircraft. 

The same type of analysis can be applied to the individual radio 
chassis. An expectation of 0.84 removal during the 3 months was the 
appropriate measure. Four or more removals of a particular radio 
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would be expected to occur on only one radio during the time period, 
but five such radios were observed, suggesting that individual serial 
number radios also had a problem. 

An even further examination of this particular field problem 
involves the individual failure modes. The tabulation below shows 
that two adjustment jobs constitute a significant percentage of all of 
the problems encountered in these sets during the first quarter of 
1977. The other principal contributor, a transistor, is now known to 
be the result of a deficient specification in the original manufac- 
ture; a campaign is now being conducted in the field to replace all of 
these transistors. The last two entries in the table are probably not 
epidemic problems. 

REPAIRS INDICATING PREVALENT FAILURE MODES 


ADJUST AUDIO 9 
REPLACE Q208 9 
ADJUST SIDETONE 8 
REPLACE Q604 4 
REPLACE Q410 3 


Total 33 = 50 percent of 

all repairs 

Another of the clear-cut issues in this instance is the effect of 
ancillary equipment on total system reliability. Many reliability 
analyses do not include the ancillary equipment on the basis that it 
is presumed to be an unimportant factor. However, experience with 
current avionics equipment indicates that this is simply not the 
case.* For example, during the first quarter of 1977, the cargo car- 

*It was not the case more than 10 years ago, as already shown in 
Table III-l. 
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rier mentioned previously had the following experience with the com- 
munications subsystem of their aircraft: 


MAINTENANCE EXPERIENCE: (FIRST QUARTER 1977) 


' ITEM 

REMOVALS 

PERCENT 

MTBF 

HEADSETS 

60 

28% 


AUDIO PANEL 

53 

24% 

(600 HRS) 

RECEIVER-TRANSMITTER 

78 

36% 


MICROPHONES 

25 

12% 



-QUANTITY PER AIRCRAFT 


In the table above, failure information for the voice recorder 
was omitted, since it is not an essential part of the system. Note 
that the audio panels accounted for 24 percent of the removals during 
the 3-month time period. Their MTBF was approximately 600 hours, the 
lowest MTBF of any item in the assembly, even during a time when the 
receiver-transmitters were an epidemic problem. Note also that head- 
sets and raicrophones--clearly ancillary equipment--made significant 
contributions to removal rates during the same time period. 

In summary, then, general aviation avionics reliability does not 
seem to differ much from the reliability characteristics derived for 
other classes of electronic equipment, nor for that matter from avion- 
ics reliability in military systems. 

Given that reliability estimates based on actual records from 
general aviation avionics usage are hard to come by, one can reason- 
ably ask what other tools might be brought to bear. One obvious 
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answer is the previously described reliability prediction work, par- 
ticularly MIL Handbook 217B. In that specific case, some precautions 
should be observed. We know, for example, that MIL Handbook 217B has 
a bias toward a higher reliability than is usually justified in 
estimating. Also, commercial parts are normally used in general avia- 
tion avionics equipment, rather than MIL spec parts as customarily 
assumed in a 217B analysis. Furthermore, the 217B approach assumes a 
relatively benign environment in general, even though an allowance is 
made for being airborne in a manned environment as an example. 
Finally, less than optimal installations frequently occur in the gen- 
eral aviation avionics business, and these are known to have a ten- 
dency to reduce inherent reliability [72] . If the above factors are 
not considered in a reliability analysis, the outcome will indicate an 
unreasonably optimistic reliability prediction. 

If a system design is predominantly the assembly and interconnec- 
tion of sub-units that are in an initial production phase or in a 
mature design and usage stage, then estimation of reliability of the 
final system offers far fewer challenges, at least for an initial 
evaluation. Under these circumstances , field reliability estimates 
for individual components are entirely appropriate, though somewhat 
difficult to obtain. Assuming that such estimates can be obtained 
somehow, there are still some limitations which should be considered 
as applicable to the reliability figures thus derived. The assumption 
is inherent in most field reliability estimates that the highest qual- 
ity of maintenance will be available, and this frequently is not the 
case. Another assumption will be that no untested installations will 
be represented, and this also is seldom true in general aviation 
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equipment. Long flights are likely to make disproportionate contribu- 
tions to the reliability data bank available to anyone looking for 
such information, and these are not typical of general aviation avion- 
ics. Likewise, heavy use will be a characteristic of those data 
sources which will constitute the bulk of the information. Always, an 
allowance must be made for interface problems, which are highly likely 
to be a significant factor, and may even dominate the reliability pic- 
ture for such an avionics assembly. Finally, to be fair, we should 
note that flight reliability should be better than the customary MTBF 
calculations show, because of the turn-on stress mentioned earlier. 

One rather novel approach has been developed for estimating 
avionics reliability, with at least some degree of success. Many 
researchers had previously found that support costs derived from 
maintenance activities can be approximated for a fair variety of 
equipment as a few percent of original equipment cost per year [45]. 
This can be translated into a method for estimating the reliability of 
prototype or early production systems, and all such methods seem to 
have flowed from a study done at The Rand Corporation 25 years ago 
[78]. As an example, an ARPA study done a few years ago noted that 
for nearly fifty Air Force avionics units comprising communication, 
navigation, and computer functions, a fair approximation of field 
reliability was provided by the single measure of unit cost, in a sim- 
ple inverse relationship: 

6 

1.3 x 10 

MTBF = 


DOLLAR COST 
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Obviously, very cheap or very expensive equipments may deviate consid- 
erably from this relation. The study also examined techniques by 
which certain special equipments had mean times between failure one to 
two orders of magnitude (i.e., 10 to 100 times) higher than more 
representative units [74]. 

CONCLUSIONS AND RECOMMENDATIONS 

In this section, we have tried to illustrate that both the pred- 
iction of the reliability of future systems and assessment of the 
reliability of existing systems pose significant problems for the 
unwary. 

Reliability predictions tend to be severely biased toward the 
high side, mainly by various factors associated with competition, but 
also by typically novel aspects in new designs: new and untried cir- 
cuits, new parts instead of standard, etc. All of these make the 
prediction of future numerical reliability extremely hazardous. How- 
ever, the prospects for a more limited objective in this area — the 
comparison of two (or more) candidate designs for their inherent rela- 
tive reliability--are really quite favorable, given only few easily 
understood precautions. First, use of the standard MIL Handbook 217B 
methodology ensures that many factors known to affect reliability are 
assessed equally between or among candidates. The use of a newly 
available digital computer program for this purpose increases the 
assurance of completeness and lack of bias (both essential to produc- 
ing the best of what may still be flawed predictions) , and at the same 
time reduces the effort and expense required to provide this vital 
comparative measure. 
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The specific procedures consist of: 

1. Listing the individual electronic parts of the system; 
this means that a detail design must exist. 

2. Associating all relevant parameters with each part,, 
such as operating conditions, quality level, etc. 

3. Identifying special characteristics of parts (number of 
gates in an IC, voltage rating of capacitors, etc.). 

4. Applying the above information to tables in MIL 
handbook 217B. 

This detailed procedure does not mean that these most modern 
evaluation methods cannot be misused to provide extravagant reliabil- 
ity estimates. They can, by the relatively simple expedient of minim- 
izing initial design complexity to the point where the later addition 
of circuits and components will become necessary in order to meet per- 
formance requirements. Even higher reliability estimates may be had 
through optimistic parts selection and derating policies. It should 
be clear that such unrealistic assumptions are less likely to be 
accepted now, given today's standardized estimating tools. * 

A second appropriate course of action should be a continuing sur- 
veillance of the current state of avionics reliability in general avi- 
ation. The assessment of current avionics reliability is made diffi- 


*A related problem arises with regard to comparing systems in 
different stages of development, as for example the SIU and STI/MILCO 
PCAAS proposals, the former a prelimnary design and the latter mainly 
an assembly of off-the-shelf units. 
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cult primarily by the substantial number of factors known to affect 
it, and the lack of representative data on GAA usage, the latter being 
caused by the understandable reticence of those holding even imperfect 
information on the subject to share it with the world. This situation 
is relieved somewhat by the growing existence of experience data banks 
on reliability among users of equipmment as opposed to manufacturers, 
but because these users tend to be large, they represent the more 
favorable side of environment, installation, and maintenance, and even 
of length of flight and intensity (hours/day) of use. 

A third appropriate reaction is to watch for relevant experience 
with other integrated (multiplexed) systems , particularly commercial 
and Air Force avionics. Expanded activities in these areas should 
make more reliability information available on that subject. 

Needless to say, sufficient problems remain to cause even profes- 
sional reliability engineers to have a conservative view of what can 
be accomplished. Two of the more outstanding areas in which informa- 
tion is inadequate are those relating to interface problems, which 
have yielded only somewhat to accepted standards in (for example) mul- 
tiplex systems, and software reliability, which becomes a questionable 
area in virtually every new system.* 

If any one recommendation is worth making about this subject, it 
is to beware of false prophets, who have been around since reliability 
emerged as a discipline. There are no panaceas, and only a relatively 


*The second edition of the well known "Reliability: Management, 
Methods and Mathematics" by David Lloyd and Myron Lipow has a long 
chapter devoted to this subject [79]. 
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limited (though increasing) number of viable tools that, when used 
with appropriate care, can often deliver enlightening results. 

Finally, we have observed that quality of maintenance has 
demonstrably beneficial effects on reliability as reflected in mean 
times between failures. Some implications of this fact are explored 


in the next section. 
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IV. MAINTAINABILITY OF ADVANCED AVIONICS SYSTEMS 
INTRODUCTION 

The subject of maintainability for general aviation avionics 
equipment has been of deep concern to manufacturers, service shops, 
and professional organizations (for example, the Aviation Electronics 
Association (AEA)) for many years. Through the efforts of these 
organizations maintainability of general aviation avionics equipment 
has been improving, but avionics maintenance is still a significant 
part of the cost of ownership. 

In promoting the development of highly integrated avionics sys- 
tems for general aviation aircraft, the problem of maintainability is 
understandably of deep concern to NASA. This concern stems partly 
from the known problems and costs of maintaining contemporary equip- 
ment, and partly from the fact that in the few instances in which com- 
ponents have been interconnected in an architecture similar to that 
shown in Fig. I, the problems of fault isolation have been found to be 
very different from those encountered in more conventional systems. 
The capability of incorporating a range of self-test programs in these 
systems is appealing, in the expectation that it could greatly sim- 
plify fault isolation. However, a significant incorporation of these 
features in an operating system is yet to be completely successful. 

The initial objective of the maintainability part of this study 
was to develop a generalized methodology that would allow the predic- 
tion of maintenance costs (man-hours/flight hours) as a function of 
desiga/performance parameters. The development of this methodology 
depended on the availability of field experience data on avionics from 
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the general aviation community, and our ability to identify the per- 
tinent design/performance parameters. 

During the course of the study, we attempted to locate data and 
identify design/functional parameters through interviews throughout 
the avionics industry. We found that field data on maintenance are 
equally as scarce as cost and reliability data. Although some FBO’s 
keep records on maintenance action, the cost of assembling these data 
is prohibitive. It was also difficult to ascertain if there were 
enough data in the records to permit reasonable statistical analysis. 

Although the study was unable to produce generalized methodolo- 
gies, the interviews with the many organizations and individuals pro- 
duced considerable information that can be useful to anyone designing 
avionics equipment for maintainability, and for NASA to judge the 
maintainability of a proposed design. 

In this section, we have assembled the results of our industry 
interviews. We have tried to identify and, to the extent possible, 
analyze the many factors that might be useful in evaluating or design- 
ing an advanced avionics system. In trying to piece together the 
parameters affecting the cost of ownership, one is quickly made aware 
that maintainability is a complicated concept involving such varied 
factors as quality of service people, service organizations, equipment 
design, documentation, usage environment, test equipment, and test 
procedures. 

In discussing maintenance with a number of users, it is clear 
that they have more than a passing concern about avionics maintenance. 
On the average, it has been estimated that the maintenance bill for 
contemporary solid-state general aviation avionics equipment runs 



-81- 


about 3.5 percent of the original cost per year [45]. This does not, 
however, represent the total cost picture, as the user experiences it. 
In many cases it requires 2 or 3 days to troubleshoot and repair a 
complicated unit, and the aircraft is usually unavailable for use dur- 
ing this period. Consequently, the knowledgeable purchaser of a new 
avionics suite is not only looking for advanced capability, but also, 
as a concomitant goal, for the reduction of the maintenance costs and 
the mean time to repair (MTTR) . 

Most of the documented experience in designing for maintainabil- 
ity is either from the military or the commercial airlines. While 
much can be learned from looking at this experience, it should be 
recognized at the outset that the maintenance structure for supporting 
the general aviation user is very different from that of either the 
military or the airlines. In addition to organizational differences, 
the skill levels of the people, the complexity of the equipment, and 
the economics are also different. In an airline operation, economics 
dictate that a quick return of the aircraft to service is the number 
one objective, even at considerably higher maintenance cost. This 
requirement for a short MTTR also exists for military aircraft in time 
of crisis operations or combat; consequently, the military maintenance 
philosophy must also be geared to this type of operation. 

AIR FORCE EXPERIENCE 

In the Air Force operation, the user seemingly has substantial 
control over his destiny. Maintainability requirements can be written 
into procurement contracts to ensure that newly acquired equipment can 
be supported. Also, the avionics maintenance personnel are under the 
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direct control of the user. The training level of the people is set 
by the user, and if additional training is believed to be needed, it 
is provided at the user’s expense. The user also controls who repairs 
what, and where repairs will be made. Inventories of line replacement 
units (LRUs) are maintained at the expense of the user, so that an 
aircraft can quickly be returned to service with simple replacement of 
failed units. Malfunctioning units that have been removed from ser- 
vice are then repaired in the shop and returned to inventory. In the 
case of the Air Force, different levels of shop maintenance are 
authorized for different organizational entities, e.g., minor repairs 
(printed circuit board replacement, etc.) are performed at the base 
shop, and units requiring major repairs are shipped to a depot. In 
either event, each LRU is repaired by someone who has specialized on 
that particular unit. Both the Air Force and the airlines have con- 
cluded that today's avionics are so complex, and have such a multipli- 
city of functions, that it is not reasonable to provide in-depth 
training in each system for all line and shop personnel. 

In spite of having complete control over its maintenance operat- 
ing equipment, philosophy, and personnel, the Air Force is still hav- 
ing serious problems in maintaining the highly integrated advanced 
avionics systems that have entered the inventory since 1968. This has 
caused much concern, and considerable effort is being expended on 
understanding this problem. 

A 1974 Rand study [8] took a broad comprehensive look at the Air 
Force problem in developing and maintaining these highly integrated 
systems. Some of the observations and conclusions of this study may 
provide insight into approaches to designing for maintainability in 
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iategrated avionics systems for general aviation. A few of the more 
important ones are presented here. 

The following conclusions were held by the 1974 Rand study team 
members at the beginning of the study, and formed the basic premises 
for their research: 


1. In the development of a new component, there cannot be 
complete a priori knowledge of that component's failure 
modes. 

2. In the integration of known components into new arrays, new 
failure modes are introduced which cannot be completely 
predicted. 

3. The only way we know of thus far to identify these failure 
modes prior to operational deployment, and to provide 
reliable verified diagnostic procedures for subsequent 
field identification, is by long and expensive test programs. 

4. To perform technical improvements repeatedly in a new 
weapons acquisition program is expensive, time-consuming, 
and operationally imprudent. 

It was found that the interaction of the units of a highly 
integrated system sometimes produced results that were unexpected. 
For example, the report stated the following: 


Modern avionics systems are integrated through a digital 
computer; or, as in the FB-111, a group of digital 
computers receive information from the various items of 
equipment, process it, and provide corrections back to the 
various pieces of equipment plus output indications to the 
crew. Through the corrections provided to individual 
pieces of equipment, interactions occur between the pieces. 
An output from one item of equipment in an integrated 
system reflects not only its own performance but also that 
of others. Moreover, a malfunction in one item of equip- 
ment may be detected more clearly in the output from 
another. Such a coupling effect from one piece of equipment 
to another is a new phenomenon in avionics, and is one 
characteristic of highly integrated systems. 
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It was observed that the performance of an integrated system was 
determined by how closely the equations that constitute the mathemati- 
cal model of the system represented the real world and the desired 
performance. Any failure in the model produced a faulty operation, 
the same as a malfunctioning piece of hardware would cause faulty 
operation. Much of the problem experienced by the Air Force in util- 
izing integrated systems was identified as the failure of the model to 
accurately describe the functional requirement and the real world in 
all operational situations. 

Another serious problem identified by the Rand study was inade- 
quate fault-isolation capability. This aspect of the problem was 
characterized in the following way: 


The nature of fault isolation has changed under the impact 
of integrated, complex avionics systems and highly automated 
test equipment for inflight diagnosis, shop repair, and 
depot repair. No longer does a pilot describe a malfunc- 
tioning piece of equipment to ground personnel, who proceed 
to trouble-shoot the equipment. Instead the functioning of 
a built-in test equipment light signals a malfunction to 
the ground crew, who remove the indicated item of equipment 
and take it to the avionics maintenance shop. There the 
equipment is subjected to a complete standard test on 
automated test equipment. If it fails the test, the indi- 
cated plug-in modules are replaced and the test is repeated. 
If the equipment cannot be repaired, it is returned to the 
depot where automatic test equipment is used in a similar 
fashion by more experienced people. 

This growth of automatic test equipment at all echelons of 
repair standardizes the testing and reduces the training 
required to trouble-shoot the equipment; however, as the 
above description illustrates, it almost completely cuts 
off communication between the levels of maintenance and it 
places total reliance on the fault-isolation capability of 
the automatic test equipment. 

Unfortunately, the fault-isolation capability of built-in 
tests and shop test equipment has not been adequately 
tested. Current procedures only show that the test equip- 
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ment will pass good units and will fail completely faulty 
units. Since the actual distribution of failure modes is 
not known when the test equipment, its associated computer 
programs, and its operating procedures are developed, it 
is not surprising that the effectiveness of the test equip- 
ment in fault-isolating field failure modes is completely 
untested. 

In spite of this uncertainty, the capability of the built- 
in test equipment and shop aerospace ground equipment to 
isolate faults is taken for granted in the maintenance 
philosophy and in spare provisioning. This, and the lack 
of communication between levels, leads to frequent dis- 
agreement between test results at different levels of 
maintenance. A fault indication will appear on a flight, 
but subsequent shop test shows no fault with the equipment. 
Components are removed in a shop test as faulty, but in 
depot testing they show no fault. In the absence of 
special tests, these results appear only in the aggregate 
results of higher-level testing, since the lack of communi- 
cation between levels obscures results in specific cases. 

The pertinent point is that deficiencies in fault-isolation 
underlie a large part of avionics problems. To correct 
these deficiencies will require improved validation of the 
capability. In practice, extensive testing of the equip- 
ment and computer programming affecting fault-isolation is 
extremely lengthy and expensive. A possible compromise 
is to reduce the scope of the problem through the use of 
already developed and tested building-block equipment 
components. With these, the principal failure modes and 
symptoms should be well understood before the equipment 
is used in a particular system. Testing of fault-isolation 
can concentrate on the limited set of new failure modes 
induced by the particular application or environment of 
the new system. Divide and conquer, so to speak. 


The maintenance problem of integrated avionics systems in the Air 
Force has been further complicated by the inability to retain mainte- 
nance personnel. Data collected in a 1976 Rand study [6] indicated 
that in 1975, the probabiity that a person who initially entered the 
avionics flight-line maintenance career field would remain in the Air 
Force to his fifth year of service was 14 percent. 





AIRLINE EXPERIENCE 


The air carriers do not face the same complexity level as the Air 
Force. Their requirement is to get from point A to point B safely in 
all kinds of weather. Military avionics, on the other hand, has the 
additional complication of sophisticated weapons systems. The air- 
lines also maintain a more conservative approach to avionics systems. 
This conservative posture is exemplified in a number of ARINC stan- 
dards; for example, the recent standard for DITS (MARK 33 Digital 
Information Transfer System). The standard excludes multiplexed use 
of transfer buses, allowing only one information source on each bus. 
All of this is not to say the airlines do not have complex maintenance 
problems--only less so than those faced by the Air Force. 

One requirement in airline maintenance is fast turnaround time. 
The average time between arrival and departure is 1/2 hour for through 
flights, and 1 hour for turnaround. During a recent seminar on air- 
line avionics maintenance [5] , one airline official summed up the 
problem as follows: 


In the amount of time listed, we and notably the 
manufacturers expect a line mechanic to look in an 
aircraft log book, analyze a hurriedly written set 
of symptoms about a system or systems that transcend 
three or more ATA chapters in the maintenance manual, 
may involve 50 or more LRUs, and make the right 
decisions with at least 85 percent probability of 
success!! With the present troubleshooting aids, or 
lack of aids, they do well to achieve their present 
success rate. 



-87- 


The principal maintenance aids currently used by line technicians 
to isolate problems are fault balls,* BITE (Built In Test Equipment), 
inline monitors, and ground test equipment. The following evaluation 
of the quality and utility of these different diagnostic aids was 
presented by Mr. Jim Takeuchi of United Airlines at the Avionics 
Maintenance Conference : 


Fault Balls 


The airlines are strongly of the opinion that fault 
in most of today’s so equipped LRUs are ineffective 
and even misleading. Specifically, they are classified as 
"oversensitive" and "unreliable." A very few LRUs are 
referred to as having reliable and effective fault ball 
operation. A survey conducted some time ago at United 
illustrates the point: of 940 aggregate fault balls on 24 

LRUs installed in wide-bodied aircraft, 408 were tripped, 
eight of which with related write-ups and 400 without any 
write-up in the log book.** 


B uilt In Test Equipment ( BITE ) 

The airlines are almost unanimous that BITE as it 
exists today is inadequate. It is expensive. It is 
insufficiently reliable within itself, and has a very low 
confidence level. Some airlines seriously question the cost 
effectiveness of BITE. Some time ago an engineer at United 
made a study of the 727 A/P BITE reliability and came up 
with the following interesting observations : 

a. Following a gripe, if a unit self tests good, is 
the system good? - 53 percent yes. 

b. Following a gripe, if a unit self tests bad, is it 
bad? - 60 percent yes. 


*Fault balls are electromechanical indicators used to indicate 
the occurrence of an event. They can be triggered by a voltage level 
or a pulse, and once triggered, they remain triggered until they are 
reset mechanically. 

**Airline responses to a questionnaire indicated that the concept 
of fault balls itself was not in question. It is the circuit that 
should be refined to be practical. 
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c. Following a gripe, if a unit is replaced straight off, 
without self test, is the plane fixed? - 63 percent 
yes . 

Historically, BITE or self test was given up as 
something extra or a nice thing to have. It seldom 
came as a result of a solid maintenance management 
plan. It needs a systematic approach. Any self test, 
for which we pay plenty, must be considered as an 
integral part of the basic system design, not as an 
added on feature. ...We the airlines, need on board 
line maintenance aids and testing methods that are at 
least 85 percent effective in localizing a defective 
LRU, transducer relay, etc. This is a must for multi 
box systems. 


In-Line Monitor 


As the purpose of the in-line monitor is to inform 
flight crews of failure(s) detected in flight, and not 
necessarily to be used as a primary line maintenance tool, 
it provides little help for the line mechanic. Nevertheless, 
the in-flight monitor has been a source of many unconfirmed 
removals. An important point is that the monitor threshold 
levels must carefully be determined to minimize unnecessary 
removals. It is one thing to establish design tolerances; 
it is quite another to see that these tolerances closely 
relate to the level at which pilots begin to recognize 
performance degradation. 


Line - Test Equipment 

Eighty percent of the airlines feel that today test 
boxes are so complex or inconvenient that they cannot 
practically be used except on layovers, and that simple 
inexpensive function no/go go test boxes would help isolate 
quickly to the LRU level. 

The airlines are in total agreement that improvements must be 
made in fault-isolation methods. The current shotgun approach to mak- 
ing rapid repairs (replace everything that could possibly cause the 
problem) is too costly. The capital cost of purchasing enough addi- 
tional spares is tremendous, and the shop costs to constantly test and 
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return these spares to stock further increase the cost. In addition, 
the wear and tear on plugs, connectors, and LRUs under constant remo- 
val and reinstallation ultimately cause further problems. The number 
of nonconfirmed removals (NCRs) now runs as high as 50 or 60 percent, 
and promises to become worse with the addition of more complicated 
equipment, unless better methods of fault isolation are developed. 
Unfortunately, if an airplane must be back in the air in a half hour 
or so, the shotgun approach is currently the only effective solution. 
If repeated squawks followed by a maintenance action are required to 
ultimately find a single failure in an LRU , then it becomes clear that 
reliability moves down on the importance scale , and maintainability 
becomes the dominant issue . 

In summary, airline experience in avionics maintenance shows the 
number one issue to be fault isolation in the aircraft system. 
Present methods are woefully inadequate. The next level of importance 
is one of LRU maintainability. The 1977 ARINC Avionics Maintenance 
Seminar [5] identified, and discussed in detail, a number of areas 
that need improvement: piece parts, printed circuit boards, connec- 
tors, thermal design, component accessibility, testing methods, etc., 
and, finally, the need for better documentation. Unfortunately, docu- 
mentation has not kept up with the sophistication of the new equip- 
ment. A maintainability questionnaire, consistin of 24 detailed ques- 
tions, was answered by 40 different airlines, a summary of which is 
included in the Proceedings of the ARINC Maintenance Seminar. Since 
many of the issues are equally relevant to general aviation avionics, 
the summary of this airline response to the maintainability question- 
naire should be referenced by anyone designing advanced general avia- 
tion avionics equipment. 
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GENERAL AVIATION 

There is very little documented experience on maintainability of 
general aviation avionics, and even if there were, it is doubtful that 
it would give useful insights into the problem of maintenance in 
highly integrated systems. The best source of information is the mil- 
itary and airline experience previously discussed; however, the 
differences in the requirements and the organization must be kept in 
mind at all times. 

The general aviation user has no standard procedure for maintain- 
ing equipment. Most users rely on the independent avionics repair 
shop for service; however, some larger FBOs do maintain and control 
their own independent facilities and personnel. 

All shops performing service on installation must be certified by 
the FAA. The rules governing the shop operation and certification are 
set down in Civil Aeronautics Manual 52. CAM 52 requires that a shop 
have (a) appropriate service and instruction manuals issued by the 
manufacturer, and (b) minimal prescribed electronic test equipmment. 
Also, the individual in charge of inspection, maintenance, and 
overhaul must have had 18 months experience in the work he is 
supervising. The FAA currently divides avionics equipment into three 
classes: communications equipmment, navigation equipment, and radar 
equipment. A shop must have certified personnel and equipment for 
each class of equipment that it will install or service. 

On the surface, these minimal safeguards seem enough; but in 
practice, they afford very little help to the user of the service. It 
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is very difficult for a general aviation user to make an a priori 
evaluation of a particular independent shop, or of the individual 
capabilities within a shop. One shop may be capitalized at $30,000, 
which probably represents the bare minimum of equipment and material, 
whereas another shop may have $200,000 or $300,000 in sophisticated 
test equipment. Also, the FAA requires only that the individual in 
charge be certified; the qualification of the remaining service people 
is the responsibility of the management. A requirement of 18 months 
of practical experience for a supervisor is very little assurance of 
competence. Also, the required management certification of competence 
adds very little to the user's confidence level. 

Competency in service personnel is one of the most serious prob- 
lems faced by the general aviation community. Many of the service 
technicians stay with general aviation only long enough to build up 
experience; then they move into airline jobs where the pay is higher, 
the fringe benefits are better, and the working hours and conditions 
are better. There are some good avionics shops throughout the coun- 
try, and there are some highly qualified competent people in the busi- 
ness, but discussions with a number of FBOs who depend on independent 
avionics shops point to the general inadequacy of this service for the 
general aviation community. 

Another area where general aviation is different is in the con- 
cept of an LRU. In an Air Force or airline operation, all replacement 
units are owned by the user, and replacing an LRU is simply a matter 
of exchanging something the user owns for something else he owns. 
This is seldom the case in general aviation. A typical aircraft owner 
would be reluctant to accept the exchange of functional black boxes as 
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a means of repair unless the units he received were identical and in 
as good condition as those he originally bought. Also, the service 
shop could not possibly afford to stock serviceable units of all the 
myriad different makes and models of equipment he is likely to 
encounter. The maximum LRU level of complexity that seems acceptable 
to the average owner, and that would be reasonable for most avionics 
shops, is the printed circuit (PC) board level. The PC board as a 
field replacement unit has gained wide acceptance in repairing home 
television sets, computers, computer terminals, and other electronic 
equipment. 

In designing new systems, it is the responsibility of the 
manufacturer to decide on the maintenance philosophy to be followed. 
Once that philosophy is established, equipment design and factory sup- 
port must conform to that philosophy. For example, if the principal 
LRU is to be the printed circuit board, a number of things must be 
done. The system design must incorporate pluggable PC modules, the 
modules should be accessible without removal of major chassis, and the 
modules should conform to system function in such a way that they will 
support fault diagnosis. System partitioning into LRUs can be devised 
in such a way that it will minimize the time required for fault diag- 
nosis (and thus, support costs), provided this aspect of system design 
is considered simultaneously with other system requirements and 
maintenance planning. 

It is highly unlikely that avionics manufacturers can provide, or 
influence to any significant degree, the incentives needed to attract 
good, high-level people to the general aviation avionic maintenance 
career field. Also, any pressures applied by the FAA in the form of 
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more rigid certification requirements will only serve to reduce the 
number of people who choose avionics maintenance as a career, assuming 
that salaries remain the same. This is unfortunate, but it is a fact 
of life that must be reckoned with. Another problem is that contem- 
porary avionics maintenance personnel are trained and experienced 
predominately in analog devices, with little or no knowledge of digi- 
tal circuits and systems. Although training and exposure over time 
will change this, there is no way to quickly implant many years of 
experience into the profession. 

The ability of maintenance personnel is an additional factor that 
must be considered by the manufacturer in establishing maintenance 
philosophy. A reasonable approach is to conduct research and experi- 
mentation on job performance aids and training methods to enable the 
utilization of persons of lower levels of ability. This approach has 
been used by the television service industry and to some extent, the 
computer industry, with reasonable success. 

Another part of the maintenance philosophy relates to the kind of 
support required from the manufacturer. If the intended philosophy is 
that PC boards are not normally repaired by the independent avionics 
repair shop, then exchange serviceable boards should be available to 
the shops within 24 hours or less. Since shipping time is an impor- 
tant factor, regional resupply or repair stations may be required. 
Regional factory repair stations could provide complete maintenance 
service to the user; however, the impact of this competitive posture 
with the independent shops should be examined carefully. Advanced 
avionics systems that are highly integrated, such as the Demonstration 
Advanced Avionics System (DAAS), offer some unique challenges in the 
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area of maintainability. There is some military experience, and there 
is some experience from other industries; however, the applicability 
of this experience to general aviation is not completely clear. For 
this reason, the maintenance philosophy and the means for carrying it 
out must remain flexible to accommodate what is learned about the real 
problems and the effectiveness of the system to deal with them. This 
means that information feedback to the manufacturer must be planned 
and provided for. One means of keeping the manufacturer in the infor- 
mation feedback loop longer is by providing extended warranty services 
(beyond 1 year). This would discourage unfranchised installations, 
and encourage factory-controlled maintenance. Extended warranty ser- 
vice contracts have been used successfully by RCA in television war- 
ranties, and by Sears Roebuck & Company in the maintenance of home 
appliances . 

CONCLUSIONS AND RECOMMENDATIONS 

The Air Force and airline experience with advanced avionics 
equipment shows fault diagnosis and isolation to be the number one 
maintenance problem. One of the problems that frustrates the design 
of tests (i.e., diagnostic program development) for detection and 
location of faults is that when systems are highly integrated, failure 
modes are introduced that are very difficult to predict. For this 
reason it is essential to emphasize fault isolation early in the 
design stages. To ensure that maintainability is considered early in 
the design, a requirement must be placed on the designer to establish 
a maintenance philosophy and plan as part of the preliminary design. 
An evaluation of the philosophy and plan should be conducted at each 
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design review to ascertain if they will work, given the technical 
capabilities of existing maintenance personnel and the organizational 
structure. 

A part of the early design should include a model describing the 
equipment performance; however, since real world performance cannot be 
predicted precisely, it is important to continue the testing and 
updating of this model throughout the design, development, prototype 
testing, and early operational phases of the program. 

Although the contemporary approach is toward automatic fault 
diagnosis, there is still a requirement for some degree of human 
interaction in the operation of tests, and in the interpretation of 
test results. This human interaction in the fault diagnosis process 
must also be tested during the development process. 

In testing of human interaction with the fault diagnostics, the 
technical capabilities of the individuals involved should be carefully 
considered. To utilize design engineers with a deep familiarity with 
the intricacies of the system and the automatic test procedures (ATP) 
will reveal very little about how a line service technician can use 
the ATP for fault isolation. Testing must be representative of the 
real world. 

The opinion of people who have responsibilities for the mainte- 
nance of avionics is that the PC board level is probably the best 
level for a line replaceable unit (LRU) in the kind of electronics 
that will be used in advanced avionics systems. There are strong 
indications from other industries (e.g., television, computer, etc.) 
that for much of the electronic equipment, the PC board is a good 
modular level to establish as an LRU. It may be that a larger unit 
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will have to be removed from the airplane and shop tested, but even 
then, the actual replacement part probably should be a PC board. It 
must be recognized that these other parts of the avionics structure 
that are not PC boards are still contributors to equipment failures. 
These components must each be analyzed and provided for in the overall 
fault diagnosis procedure and maintenance plan. 
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V. SOME SOFTWARE CONSIDERATIONS 

We have chosen to discuss software separately in this study in 
order to give it special emphasis; however, this is not to imply that 
software should be considered separately. Software and hardware are 
closely interrelated components in a system, and are not individual 
entities to be developed separately, using different management and 
control techniques. The computer and its associated software are but 
one element of the system and thus should be treated as a holistic 
subsystem during most phases of the design process. 

Any of the proposed advanced avionics systems (AAS) will clearly 
be increasingly reliant on larger and more sophisticated computer pro- 
grams to perform many of the basic functions. These may be automatic 
functions, such as the monitoring of aircraft systems, or they may be 
operations requested by the pilot. In addition to performing in- 
flight operations, computer programs may play an important role in 
fault diagnosis and in the maintenance of the avionics system itself. 
Because of the safety-critical nature of many of the in-flight opera- 
tions, as well as the potential for reducing system maintenance costs 
through fault isolation, it is important that avionics system software 
reliability and maintainability factors be carefully considered 
throughout the entire system development process. This section 
discusses some of the problems that have been experienced in develop- 
ing software, and some generally accepted guidelines for minimizing 
these problems. 

Traditionally, software has been viewed as completely reliable 
once it becomes operational, simply because its performance does not 
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degrade over time or through repeated use. However, experience has 
shown [79] that software can perform to an acceptable level most of 
the time, even though it contains coding or design errors in seldom- 
exercised logic paths. This is especially true of complex systems. 
Performance of this kind might be adequate if one were willing to 
accept degraded performance or could tolerate occasional errors. But 
in applications involving flight safety, the loss of some key function 
or faulty operation is intolerable. Experience has also shown that 
software flaws may not show up in the testing phase, and a failure may 
first appear months or even years after the system is placed in opera- 
tion. 

Lloyd and Lipow [79] present twenty-four definitions of software 
quality characteristics. Two that are critical to AAS software are 
included here to clarify the goals of good design. 


Reliability: Code possesses the characteristic 

reliability to the extent that it can 
be expected to perform its intended 
functions in a satisfactory manner. 

This implies that the program will 
compile, load, and execute, producing 
answers of the requisite accuracy; and 
that the program will continue to operate 
correctly, except for a tolerably small 
number of instances, while in operational 
use. It also implies that it is complete 
and externally consistent. 

Maintainability: Code possesses the characteristic 

maintainability to the extent that it 
facilitates updating to satisfy new 
requirements or to correct deficiencies. 

This implies that the code is under- 
standable, testable, and modifiable, 
e.g., comments are used to locate sub- 
routine calls and entry points, visual 
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search for location of branching 
statements and their targets is 
facilitated by special formats, or the 
program is designed to fit into avail- 
able resources (memory) with plenty of 
margin to avoid major redesign. 

There are differences between software and hardware reliability 
and maintainability, and these need to be well understood in assessing 
a particular design effort. For example: 

1. Software, unlike hardware, does not wear out or 
degrade over time or with use. However, software 
may "fail" (i.e. , not perform according to 
specifications) because of a hidden flaw in the 
program design or a mistake in coding. 

2. New imperfections are not introduced in making 
copies of a computer program, but existing errors 
are reproduced faithfully. 

3. Computer programs are unconstrained by physical 
laws and therefore are not susceptible to a priori 
proof that design objectives are impractical. 

4. There are many more distinct configurations and 
paths to check in software than in hardware. 

5. Software failure modes are usually different from 
hardware failure modes. Software will fail without 
warning and may leave no indication that a failure 
has occurred, or of the time and source of the 


failure . 
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Another difference is that people, including those who manage 
system development, have a much better intuitive understanding of 
hardware than they do of software. Thus, software problems might 
sometimes be overlooked until it is too late to take reasonable 
corrective action. System managers and developers of software must be 
aware of the characteristics of good software design, and how to 
achieve them during the different phases of development. 

It is clear that the way to produce reliable software is to 
remove all errors from the software package prior to placing it in 
operation; however, this is extremely difficult to do in actual prac- 
tice, It means producing software that has minimal errors in the 
first phase, and then testing and debugging it until all remaining 
errors are eliminated. In large software systems, several years of 
use may be required to find all the mistakes and bugs. 

One study [90] showed that over 60 percent of software errors 
occur during the requirements formulation, preliminary design, and 
detailed program design phases, and that less than 40 percent of them 
arise in the programming and coding phase. From this, one can see 
that the basic fundamentals of error prevention are simple: a func- 
tional structure that represents realistic requirements, and care in 
producing the system. It is extremely difficult, however, to state 
valid, complete, and unambiguous requirements for a new system. 
Therefore, any requirements that are developed must be subjected to 
vigorous scrubbing by some adversary process involving experienced 
system users to test their validity and completeness. Once a good set 
of requirements has been achieved, the design and coding of the system 
must proceed with great care. Some generally accepted techniques for 
minimizing errors in the design and coding are: 



1. The use of a hierarchical approach (e.g., top-down 
structured programming) with parallel refinement of 
functional requirements. 

2. The use of higher order languages (HOLs) unless assembly 
language is necessary to minimize storage and for timing 
problems . 

3. The use of organizational structures conducive to 
effective software production, e.g., "Chief Programmer" 
concept [91] . 

MS software will require attention throughout the lifetime of 
the system for a number of reasons. First, there will be errors to 
correct, even after a program reaches the field, and more effective 
methods will be developed for performing certain functions. Second, 
as the system matures in the field, enhancements and new functions 
will be added to meet specific new operational requirements. These 
additions will be especially important to commercially produced sys- 
tems that must be competitive with improved products from other 
manufacturers. For this reason, a plan must be devised early in the 
design phase to allow for a software maintenance activity. 

To support program changes, software documentation must be main- 
tained throughout the life of the system and a strict procedure for 
updating it must be established. This procedure should (1) ensure 
that corrections made to one copy of the documentation are reflected 
in all copies; (2) allow evolution from one system release to another 
to occur without affecting system operations; and (3) ensure that all 
changes to a system are thoroughly tested before the system is sent to 


the field. 



- 102 - 


r 


Maintainability is a feature that must be planned for in the sys- 
tem design. Some general guidelines that have been found effective 
are : 

1. Use a modular design approach. 

2. Limit the size of each module and limit the 
functional responsibility of each module. 

3. Use a coding structure that is easily understood. 

4. Include complete and consistent comments in the 
code, 

5. Prepare documentation consistent with internal 
code comments. 

C ONCLUSIONS AND RECOMMENDATIONS 

In this section, we have discussed, briefly, some important but 
often overlooked factors in the development of systems with a substan- 
tial software component. Software problems are frequently traceable 
to the early conceptual stage, and often result from poor and unreal- 
istic system requirement specifications. Failure to employ accepted 
software engineering techniques in producing the system design can 
also cause problems. Finally, errors frequently go undetected during 
the coding and integration phase because of inadequate testing and 
poor documentation. 

To prevent these problems, management and designers should assess 
the software system and its components throughout the development pro- 
cess in terms of the following critical questions: 




1. Have the requirements been scrubbed and subjected 
to an adversary process? 

2. Have hardware and software tradeoffs been made? 

3. Are good software engineering procedures being 
included in the design process? 

4. Does the design allow for easy modification and 
for update of the software? 

5,. Does the design facilitate testing, and is there 
an integrated test plan? 

6. Is documentation and documentation maintenance 
being planned for? 

It is not only important to ask these questions, but it is neces- 
sary that someone be able to accurately interpret the answers. For 
this reason, some software development expertise should be available 
in all design reviews. Although structured programming and Chief Pro- 
grammer organization are recommended, it should be emphasized that 
they will not offset the effect of poor programmer motivation and 


talent. 
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Appendix A 

ORGANIZATIONS CONTACTED DURING STUDY 


I. Manufacturers of General Aviation Equipment 

o Bendix Avionics, Ft. Lauderdale, Florida 
o Collins Avionics, Cedar Rapids, Iowa 
o RCA Avionics, Van Nuys , California 
o King Radio, Olathe, Kansas 

o Narco Avionics, Ft. Washington, Pennsylvania 
o Delco Avionics, Santa Barbara, California 

II. Manufacturers of General Aviation Aircraft 

o Cessna Aircraft, Wichita, Kansas 
o Beech Aircraft, Wichita, Kansas 

III. Users of Avionics Equipment 

o Federal Express Corporation, Memphis, Tennessee 
o Krueger Aviation, Santa Monica, California 
o IFR Electronics, Van Nuys, California 
o United Airlines, San Francisco, California 
o ARINC Incorporated, Annapolis, Maryland 
o ARINC Research, Annapolis, Maryland 

IV. Air Force Agencies 

o Rome Air Development Center, Rome, New York 
o Air Force Avionics Lab, Wright-Patterson AFB, Ohio 
o F-16 Program Office, Wright-Patterson AFB, Ohio 
o Advanced Airborne Command Post Office, L.G. Hanscom 
Field, Massachusetts 

V. Government Contractors 

o McDonnell Douglas, St. Louis, Missouri 

o Dynamics Research Corporation, Wilmington, Massachusetts 
o Hughes Aircraft Company, Culver City, California 
o Charles Stark Draper Labs, Cambridge, Massachusetts 

VI. Commercial Companies 

o INTEL Corporation, Sunnyvale, California 
o Delco Corporation, Milwaukee, Wisconsin 
o Hewlett Packard Corporation, Palo Alto, California 
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VII. Professional Organizations 

o Aircraft Owners and Pilots Association (AOPA) 
o National Business Aircraft Association (NBAA) 
o General Aviation Manufacturers Association (GAMA) 
o Aircraft Electronics Association (AEA) 
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